stsci_logo

NIRCam Imaging Pipeline Notebook#

Authors: B. Hilbert, based on the NIRISS imaging notebook by R. Diaz
Last Updated: November 18, 2024
Pipeline Version: 1.15.1 (Build 11.0)

Purpose:
This notebook provides a framework for processing generic Near-Infrared Camera (NIRCam) Imaging data through all three James Webb Space Telescope (JWST) pipeline stages. Data is assumed to be located in a folder structure following the paths set up below. It should not be necessary to edit any cells other than in the Configuration section unless modifying the standard pipeline processing options.

Data:
This example is set up to use an example dataset is from Program ID 2739 (PI: Pontoppidan) which is a Cycle 1 Outreach program. We focus on the data from Observation 001 Visit 002, in which M-16, or the “Pillars of Creation” were observed. Example input data to use will be downloaded automatically unless disabled (i.e., to use local files instead).

JWST pipeline version and CRDS context:
This notebook was written for the calibration pipeline version given above. It sets the CRDS context to the latest context in the JWST Calibration Reference Data System (CRDS) associated with that pipeline version. If you use different pipeline versions or CRDS context, please read the relevant release notes (here for pipeline, here for CRDS) for possibly relevant changes.

Updates:
This notebook is regularly updated as improvements are made to the pipeline. Find the most up to date version of this notebook at: https://github.com/spacetelescope/jwst-pipeline-notebooks/

Recent Changes:
Sept 5, 2024: original notebook created
Nov 11, 2024: Comment out line to set the context
Nov 18, 2024: Do not require both SW and LW user-provided data
November 22, 2024: Updates to workflow when skipping pipeline modules


Table of Contents#

  1. Configuration

  2. Package Imports

  3. Demo Mode Setup (ignore if not using demo data)

  4. Directory Setup

  5. Detector1 Pipeline

  6. Image2 Pipeline

  7. Image3 Pipeline

  8. Visualize the resampled images

  9. Visualize Detected Sources

  10. Notes


1. Configuration#


Set basic configuration for runing notebook.

Install dependencies and parameters#

To make sure that the pipeline version is compatabile with the steps discussed below and the required dependencies and packages are installed, you can create a fresh conda environment and install the provided requirements.txt file:

conda create -n nircam_imaging_pipeline python=3.11
conda activate nircam_imaging_pipeline
pip install -r requirements.txt

Set the basic parameters to use with this notebook. These will affect what data is used, where data is located (if already in disk), and pipeline modules run in this data. The list of parameters are:

  • demo_mode

  • directories with data

  • pipeline modules

# Basic import necessary for configuration
import os
Note that demo_mode must be set appropriately below.

Set demo_mode = True to run in demonstration mode. In this mode this notebook will download example data from the Barbara A. Mikulski Archive for Space Telescopes (MAST) and process it through the pipeline. This will all happen in a local directory unless modified in Section 3 below.

Set demo_mode = False if you want to process your own data that has already been downloaded and provide the location of the data.

# Set parameters for demo_mode, channel, band, data mode directories, and 
# processing steps.

# -----------------------------Demo Mode---------------------------------
demo_mode = True

if demo_mode:
    print('Running in demonstration mode using online example data!')

# --------------------------User Mode Directories------------------------
# If demo_mode = False, look for user data in these paths
if not demo_mode:
    # Set directory paths for processing specific data; these will need
    # to be changed to your local directory setup (below are given as
    # examples)
    user_home_dir = os.path.expanduser('~')

    # Point to where science observation data are
    # Assumes uncalibrated data in <sci_dir>/uncal/ and results in stage1,
    # stage2, stage3 directories
    sci_dir = os.path.join(user_home_dir, 'PID2739/Obs001/')

# --------------------------Set Processing Steps--------------------------
# Individual pipeline stages can be turned on/off here.  Note that a later
# stage won't be able to run unless data products have already been
# produced from the prior stage.

# Science processing
dodet1 = True  # calwebb_detector1
doimage2 = True  # calwebb_image2
doimage3 = True  # calwebb_image3
Running in demonstration mode using online example data!

Set CRDS context and server#

Before importing CRDS and JWST modules, we need to configure our environment. This includes defining a CRDS cache directory in which to keep the reference files that will be used by the calibration pipeline.

If the root directory for the local CRDS cache directory has not been set already, it will be set to create one in the home directory.

# ------------------------Set CRDS context and paths----------------------

# Each version of the calibration pipeline is associated with a specific CRDS
# context file. The pipeline will select the appropriate context file behind
# the scenes while running. However, if you wish to override the default context
# file and run the pipeline with a different context, you can set that using
# the CRDS_CONTEXT environment variable. Here we show how this is done,
# although we leave the line commented out in order to use the default context.
# If you wish to specify a different context, uncomment the line below.
#%env CRDS_CONTEXT jwst_1293.pmap

# Check whether the local CRDS cache directory has been set.
# If not, set it to the user home directory
if (os.getenv('CRDS_PATH') is None):
    os.environ['CRDS_PATH'] = os.path.join(os.path.expanduser('~'), 'crds')
# Check whether the CRDS server URL has been set.  If not, set it.
if (os.getenv('CRDS_SERVER_URL') is None):
    os.environ['CRDS_SERVER_URL'] = 'https://jwst-crds.stsci.edu'

# Echo CRDS path in use
print(f"CRDS local filepath: {os.environ['CRDS_PATH']}")
print(f"CRDS file server: {os.environ['CRDS_SERVER_URL']}")
if os.getenv('CRDS_CONTEXT'):
    print(f"CRDS CONTEXT: {os.environ['CRDS_CONTEXT']}")
CRDS local filepath: /home/runner/crds
CRDS file server: https://jwst-crds.stsci.edu

2. Package Imports#

# Use the entire available screen width for this notebook
from IPython.display import display, HTML
display(HTML("<style>.container { width:95% !important; }</style>"))
# Basic system utilities for interacting with files
# ----------------------General Imports------------------------------------
import glob
import time
from pathlib import Path

# Numpy for doing calculations
import numpy as np

# To display full ouptut of cell, not just the last result
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

# -----------------------Astroquery Imports--------------------------------
# ASCII files, and downloading demo files
from astroquery.mast import Observations

# Astropy routines for visualizing detected sources:
from astropy.table import Table
from astropy.coordinates import SkyCoord

# ------------ Pipeline and  Visualization Imports -----------------------

# for JWST calibration pipeline
import jwst
import crds

from jwst.pipeline import Detector1Pipeline
from jwst.pipeline import Image2Pipeline
from jwst.pipeline import Image3Pipeline

# JWST pipeline utilities
from asdf import AsdfFile
from jwst import datamodels
from jwst.associations import asn_from_list  # Tools for creating association files
from jwst.associations.lib.rules_level3_base import DMS_Level3_Base  # Definition of a Lvl3 association file

# For visualizing images
from jdaviz import Imviz

# Echo pipeline version and CRDS context in use
print(f"JWST Calibration Pipeline Version: {jwst.__version__}")
print(f"Using CRDS Context: {crds.get_context_name('jwst')}")
JWST Calibration Pipeline Version: 1.15.1
CRDS - INFO -  Calibration SW Found: jwst 1.15.1 (/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst-1.15.1.dist-info)
Using CRDS Context: jwst_1293.pmap
# Start a timer to keep track of runtime
time0 = time.perf_counter()

3. Demo Mode Setup (ignore if not using demo data)#


If running in demonstration mode, set up the program information to retrieve the uncalibrated data automatically from MAST using astroquery. MAST allows for flexibility of searching by the proposal ID and the observation ID instead of just filenames.

For illustrative purposes, we focus on data taken using the NIRCam F200W and F444W filters and start with uncalibrated data products. The files are named jw02739001002_02105_0000<dither>_nrc<det>_uncal.fits, where dither refers to the dither step number, and det is the detector name. Through this notebook we will refer to data with filter F200W as SW data and F444W as LW data.

More information about the JWST file naming conventions can be found at: https://jwst-pipeline.readthedocs.io/en/latest/jwst/data_products/file_naming.html

# Set up the program information and paths for demo program
if demo_mode:
    print('Running in demonstration mode and will download example data from MAST!')
    program = "02739"
    sci_observtn = "001"
    
    data_dir = os.path.join('.', 'nrc_im_demo_data')
    download_dir = data_dir
    sci_dir = os.path.join(data_dir, 'Obs' + sci_observtn)
    uncal_dir = os.path.join(sci_dir, 'uncal')

    # Ensure filepaths for input data exist
    if not os.path.exists(uncal_dir):
        os.makedirs(uncal_dir)
        
    # Create directory if it does not exist
    if not os.path.isdir(data_dir):
        os.mkdir(data_dir)
Running in demonstration mode and will download example data from MAST!

Identify list of science (SCI) uncalibrated files associated with visits.

Work one filter at a time, so that we can more easily filter by detector and keep only the module A files.

First download the F200W data.

# Obtain a list of observation IDs for the specified demo program
if demo_mode:
    # Science data
    sci_obs_id_table = Observations.query_criteria(instrument_name=["NIRCAM/IMAGE"],
                                                   provenance_name=["CALJWST"],  # Executed observations
                                                   filters=['F200W'],  # Data for Specific Filter
                                                   obs_id=['jw' + program + '-o' + sci_observtn + '*']
                                                   )
if demo_mode:
    sci_obs_id_table
Table masked=True length=1
intentTypeobs_collectionprovenance_nameinstrument_nameprojectfilterswavelength_regiontarget_nametarget_classificationobs_ids_ras_decdataproduct_typeproposal_picalib_levelt_mint_maxt_exptimeem_minem_maxobs_titlet_obs_releaseproposal_idproposal_typesequence_numbers_regionjpegURLdataURLdataRightsmtFlagsrcDenobsidobjID
str7str4str7str12str4str5str8str4str42str36float64float64str5str21int64float64float64float64float64float64str30float64str4str2int64str117str62str63str6boolfloat64str9str9
scienceJWSTCALJWSTNIRCAM/IMAGEJWSTF200WINFRAREDM-16ISM; Molecular clouds; Nebulae; Protostarsjw02739-o001_t001_nircam_clear-f200w274.7299074457347-13.851719782834468imagePontoppidan, Klaus M.359805.4032507949159824.937561909723221.04000000000041755.02226.0JWST Cycle 1 Outreach Campaign59871.559826262739DD--POLYGON 274.693610607 -13.78846435 274.770282437 -13.790845449 274.766206195 -13.91520072 274.689493363 -13.912818346mast:JWST/product/jw02739-o001_t001_nircam_clear-f200w_i2d.jpgmast:JWST/product/jw02739-o001_t001_nircam_clear-f200w_i2d.fitsPUBLICFalsenan219392596617131674
# Turn the list of visits into a list of uncalibrated data files
if demo_mode:
    # Define types of files to select
    file_dict = {'uncal': {'product_type': 'SCIENCE',
                           'productSubGroupDescription': 'UNCAL',
                           'calib_level': [1]}}

    # Science files
    sci_files_to_download = []
    # Loop over visits identifying uncalibrated files that are associated
    # with them
    for exposure in (sci_obs_id_table):
        products = Observations.get_product_list(exposure)
        for filetype, query_dict in file_dict.items():
            filtered_products = Observations.filter_products(products, productType=query_dict['product_type'],
                                                             productSubGroupDescription=query_dict['productSubGroupDescription'],
                                                             calib_level=query_dict['calib_level'])
            sci_files_to_download.extend(filtered_products['dataURI'])

    # To limit data volume, keep only files from visit 002, dithers 1 and 2, and only A-module
    sw_sci_files_to_download = [fname for fname in sci_files_to_download if 'jw02739001002_02105' in fname and 
                                ('nrca2' in fname or 'nrca4' in fname) and ('00001' in fname or '00002' in fname)]
    sw_sci_files_to_download = sorted(sw_sci_files_to_download)
    print(f"Science files selected for downloading: {len(sw_sci_files_to_download)}")
Science files selected for downloading: 4
# List the SW files to download
if demo_mode:
    sw_sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits']

Now repeat the process for the F444W data.

# Obtain a list of observation IDs for the specified demo program
if demo_mode:
    # Science data
    sci_obs_id_table = Observations.query_criteria(instrument_name=["NIRCAM/IMAGE"],
                                                   provenance_name=["CALJWST"],  # Executed observations
                                                   filters=['F444W'],  # Data for Specific Filter
                                                   obs_id=['jw' + program + '-o' + sci_observtn + '*']
                                                   )
if demo_mode:
    sci_obs_id_table
Table masked=True length=2
intentTypeobs_collectionprovenance_nameinstrument_nameprojectfilterswavelength_regiontarget_nametarget_classificationobs_ids_ras_decdataproduct_typeproposal_picalib_levelt_mint_maxt_exptimeem_minem_maxobs_titlet_obs_releaseproposal_idproposal_typesequence_numbers_regionjpegURLdataURLdataRightsmtFlagsrcDenobsidobjID
str7str4str7str12str4str11str8str4str42str36float64float64str5str21int64float64float64float64float64float64str30float64str4str2int64str119str62str63str6boolfloat64str8str9
scienceJWSTCALJWSTNIRCAM/IMAGEJWSTF444W;F470NINFRAREDM-16ISM; Molecular clouds; Nebulae; Protostarsjw02739-o001_t001_nircam_f444w-f470n274.7299074457283-13.851719782779508imagePontoppidan, Klaus M.359805.3566499935259824.897919722225797.863880.04986.0JWST Cycle 1 Outreach Campaign59871.559826262739DD--POLYGON 274.694911886 -13.788673111 274.770074195 -13.791626152 274.764955179 -13.914928446 274.689753041 -13.911973838mast:JWST/product/jw02739-o001_t001_nircam_f444w-f470n_i2d.jpgmast:JWST/product/jw02739-o001_t001_nircam_f444w-f470n_i2d.fitsPUBLICFalsenan89121540617131668
scienceJWSTCALJWSTNIRCAM/IMAGEJWSTF444WINFRAREDM-16ISM; Molecular clouds; Nebulae; Protostarsjw02739-o001_t001_nircam_clear-f444w274.7299074457347-13.851719782834468imagePontoppidan, Klaus M.359805.40325153564559824.937561909723221.04000000000043880.04986.0JWST Cycle 1 Outreach Campaign59871.559826262739DD--POLYGON 274.694954069 -13.788646948 274.770047607 -13.791598372 274.764924567 -13.914951606 274.68979122 -13.911998615mast:JWST/product/jw02739-o001_t001_nircam_clear-f444w_i2d.jpgmast:JWST/product/jw02739-o001_t001_nircam_clear-f444w_i2d.fitsPUBLICFalsenan89121644617131685
# Turn the list of visits into a list of uncalibrated data files
if demo_mode:
    # Define types of files to select
    file_dict = {'uncal': {'product_type': 'SCIENCE',
                           'productSubGroupDescription': 'UNCAL',
                           'calib_level': [1]}}

    # Science files
    sci_files_to_download = []
    # Loop over visits identifying uncalibrated files that are associated
    # with them
    for exposure in (sci_obs_id_table):
        products = Observations.get_product_list(exposure)
        for filetype, query_dict in file_dict.items():
            filtered_products = Observations.filter_products(products, productType=query_dict['product_type'],
                                                             productSubGroupDescription=query_dict['productSubGroupDescription'],
                                                             calib_level=query_dict['calib_level'])
            sci_files_to_download.extend(filtered_products['dataURI'])

    # To limit data volume, keep only files from visit 002, dithers 1 and 2, and only A-module
    lw_sci_files_to_download = [fname for fname in sci_files_to_download if 'jw02739001002_02105' in fname and 
                                'nrca' in fname and ('00001' in fname or '00002' in fname)]
    lw_sci_files_to_download = sorted(lw_sci_files_to_download)
    print(f"Science files selected for downloading: {len(lw_sci_files_to_download)}")
Science files selected for downloading: 2
# List the LW files to download
if demo_mode:
    lw_sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits']
# Full list the science files to download
if demo_mode:
    sci_files_to_download = sw_sci_files_to_download + lw_sci_files_to_download
    sci_files_to_download
['mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits',
 'mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits']

Download all the uncal files and place them into the appropriate directories.

Warning: If this notebook is halted during this step the downloaded file may be incomplete, and cause crashes later on!
# Download the demo data if it does not already exist
if demo_mode:
    for filename in sci_files_to_download:
        sci_manifest = Observations.download_file(filename,
                                                  local_path=os.path.join(uncal_dir, Path(filename).name))
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrca2_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca2_uncal.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrca4_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca4_uncal.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrca2_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca2_uncal.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrca4_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca4_uncal.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00001_nrcalong_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrcalong_uncal.fits ...
 [Done]
Downloading URL https://mast.stsci.edu/api/v0.1/Download/file?uri=mast:JWST/product/jw02739001002_02105_00002_nrcalong_uncal.fits to ./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrcalong_uncal.fits ...
 [Done]

4. Directory Setup#


Set up detailed paths to input/output stages here.

# Define output subdirectories to keep science data products organized
uncal_dir = os.path.join(sci_dir, 'uncal')  # Uncalibrated pipeline inputs should be here
det1_dir = os.path.join(sci_dir, 'stage1')  # calwebb_detector1 pipeline outputs will go here
image2_dir = os.path.join(sci_dir, 'stage2')  # calwebb_spec2 pipeline outputs will go here
image3_dir = os.path.join(sci_dir, 'stage3')  # calwebb_spec3 pipeline outputs will go here

# We need to check that the desired output directories exist, and if not
# create them
if not os.path.exists(det1_dir):
    os.makedirs(det1_dir)
if not os.path.exists(image2_dir):
    os.makedirs(image2_dir)
if not os.path.exists(image3_dir):
    os.makedirs(image3_dir)

Look at the first file to determine exposure parameters and practice using JWST datamodels¶

# List uncal files
uncal_files = sorted(glob.glob(os.path.join(uncal_dir, '*_uncal.fits')))
    
# Separate SW from LW files
sw_uncal_files = [uncfile for uncfile in uncal_files if 'long' not in uncfile]
lw_uncal_files = [uncfile for uncfile in uncal_files if 'long' in uncfile]

colnames = ('Instrument', 'Filter', 'Pupil', 'Number of Integrations', 'Number of Groups',
            'Readout pattern', 'Dither position number')
dtypes = ('S7', 'S10', 'S10', 'i4', 'i4', 'S15', 'i4')
meta_check = Table(names=(colnames), dtype=dtypes)

# Open example files and get metadata for display
if len(sw_uncal_files) > 0:
    sw_examine = datamodels.open(sw_uncal_files[0])
    sw_row = [sw_examine.meta.instrument.name, sw_examine.meta.instrument.filter,
              sw_examine.meta.instrument.pupil, sw_examine.meta.exposure.nints,
              sw_examine.meta.exposure.ngroups, sw_examine.meta.exposure.readpatt,
              sw_examine.meta.dither.position_number]
    meta_check.add_row(sw_row)

if len(lw_uncal_files) > 0:
    lw_examine = datamodels.open(lw_uncal_files[0])
    lw_row = [lw_examine.meta.instrument.name, lw_examine.meta.instrument.filter,
              lw_examine.meta.instrument.pupil, lw_examine.meta.exposure.nints,
              lw_examine.meta.exposure.ngroups, lw_examine.meta.exposure.readpatt,
              lw_examine.meta.dither.position_number]
    meta_check.add_row(lw_row)

# Print out exposure info
meta_check
Table length=2
InstrumentFilterPupilNumber of IntegrationsNumber of GroupsReadout patternDither position number
bytes7bytes10bytes10int32int32bytes15int32
NIRCAMF200WCLEAR18BRIGHT11
NIRCAMF444WCLEAR18BRIGHT11

The table above shows basic exposure information from the first shortwave as well as the first longwave file. When using the demo data, we confirm that the data file is for the NIRCam instrument using the F200W and F444W filters in the Filter Wheel crossed with the CLEAR filter in the Pupil Wheel. This observation uses the BRIGHT1 readout pattern, 8 groups per integration, and 1 integration per exposure. This data file is the 1st dither position in this exposure sequence. For more information about how JWST exposures are defined by up-the-ramp sampling, see the Understanding Exposure Times JDox article.

This metadata will be the same for all exposures in this observation, except for the dither position number.

# Print out the time benchmark
time_det1 = time.perf_counter()
print(f"Runtime so far: {time_det1 - time0:0.0f} seconds")
Runtime so far: 14 seconds

5. Detector1 Pipeline#

Run the datasets through the Detector1 stage of the pipeline to apply detector level calibrations and create a countrate data product where slopes are fitted to the integration ramps. These *_rate.fits products are 2D (nrows x ncols), averaged over all integrations. 3D countrate data products (*_rateints.fits) are also created (nintegrations x nrows x ncols) which have the fitted ramp slopes for each integration.

By default, all steps in the Detector1 stage of the pipeline are run for NIRCam except the ipc correction step and the gain_scale step. Note that the persistence step has been turned off by default starting with CRDS context jwst_1264.pmap. This step does not automatically correct the science data for persistence. The persistence step creates a *_trapsfilled.fits file which is a model that records the number of traps filled at each pixel at the end of an exposure. This file would be used as an input to the persistence step, via the input_trapsfilled argument, to correct the subsequent science exposure for persistence. Since persistence is not well calibrated for NIRCam, the step has been turned off in order to speed up calibration and to not create empty *_trapsfilled.fits files. This step can be turned on when running the pipeline in Python by doing:

rate_result = Detector1Pipeline.call(uncal, steps={'persistence': {'skip': False}})

or as indicated in the cell bellow using a dictionary.

As of CRDS context jwst_1155.pmap and later, the jump step of the Detector1 stage of the pipeline will remove signal associated with snowballs in the NIRCam imaging mode. This correction is turned on using the parameter expand_large_events=True. This and other parameters related to the snowball correction are specified in the pars-jumpstep parameter reference file. Users may wish to alter parameters to optimize removal of snowball residuals. Available parameters are discussed in the Detection and Flagging of Showers and Snowballs in JWST Technical Report (Regan 2023).

# Set up a dictionary to define how the Detector1 pipeline should be configured

# Boilerplate dictionary setup
det1dict = {}
det1dict['group_scale'], det1dict['dq_init'], det1dict['saturation'] = {}, {}, {}
det1dict['ipc'], det1dict['superbias'], det1dict['refpix'] = {}, {}, {}
det1dict['linearity'], det1dict['persistence'], det1dict['dark_current'], = {}, {}, {}
det1dict['charge_migration'], det1dict['jump'], det1dict['ramp_fit'] = {}, {}, {}
det1dict['gain_scale'] = {}

# Overrides for whether or not certain steps should be skipped
# skipping the persistence step
det1dict['persistence']['skip'] = True

# Overrides for various reference files
# Files should be in the base local directory or provide full path
#det1dict['dq_init']['override_mask'] = 'myfile.fits' # Bad pixel mask
#det1dict['saturation']['override_saturation'] = 'myfile.fits'  # Saturation
#det1dict['reset']['override_reset'] = 'myfile.fits'  # Reset
#det1dict['linearity']['override_linearity'] = 'myfile.fits'  # Linearity
#det1dict['rscd']['override_rscd'] = 'myfile.fits'  # RSCD
#det1dict['dark_current']['override_dark'] = 'myfile.fits'  # Dark current subtraction
#det1dict['jump']['override_gain'] = 'myfile.fits'  # Gain used by jump step
#det1dict['ramp_fit']['override_gain'] = 'myfile.fits'  # Gain used by ramp fitting step
#det1dict['jump']['override_readnoise'] = 'myfile.fits'  # Read noise used by jump step
#det1dict['ramp_fit']['override_readnoise'] = 'myfile.fits'  # Read noise used by ramp fitting step

# Turn on multi-core processing (This is off by default). Choose what fraction
# of cores to use (quarter, half, all, or an integer number)
det1dict['jump']['maximum_cores'] = 'half'

# Explicitly turn on snowball correction. (Even though it is on by default)
det1dict['jump']['expand_large_events'] = True

Run the Detector1 pipeline on all input data, regardless of filter.

# Run Detector1 stage of pipeline, specifying:
# output directory to save *_rate.fits files
# save_results flag set to True so the rate files are saved
if dodet1:
    for uncal in uncal_files:
        rate_result = Detector1Pipeline.call(uncal, output_dir=det1_dir, steps=det1dict, save_results=True)
else:
    print('Skipping Detector1 processing')
2024-12-09 15:57:30,544 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_system_datalvl_0002.rmap      694 bytes  (1 / 199 files) (0 / 716.1 K bytes)
2024-12-09 15:57:30,618 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_system_calver_0038.rmap    4.8 K bytes  (2 / 199 files) (694 / 716.1 K bytes)
2024-12-09 15:57:30,668 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_system_0037.imap        385 bytes  (3 / 199 files) (5.5 K / 716.1 K bytes)
2024-12-09 15:57:30,725 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_wavelengthrange_0024.rmap    1.4 K bytes  (4 / 199 files) (5.9 K / 716.1 K bytes)
2024-12-09 15:57:30,853 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_wavecorr_0005.rmap      884 bytes  (5 / 199 files) (7.3 K / 716.1 K bytes)
2024-12-09 15:57:30,926 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_superbias_0074.rmap   33.8 K bytes  (6 / 199 files) (8.2 K / 716.1 K bytes)
2024-12-09 15:57:30,990 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_sflat_0026.rmap   20.6 K bytes  (7 / 199 files) (41.9 K / 716.1 K bytes)
2024-12-09 15:57:31,113 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_saturation_0018.rmap    2.0 K bytes  (8 / 199 files) (62.5 K / 716.1 K bytes)
2024-12-09 15:57:31,160 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_refpix_0015.rmap    1.6 K bytes  (9 / 199 files) (64.5 K / 716.1 K bytes)
2024-12-09 15:57:31,232 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_readnoise_0025.rmap    2.6 K bytes  (10 / 199 files) (66.1 K / 716.1 K bytes)
2024-12-09 15:57:31,280 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_photom_0013.rmap      958 bytes  (11 / 199 files) (68.7 K / 716.1 K bytes)
2024-12-09 15:57:31,370 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pathloss_0007.rmap    1.1 K bytes  (12 / 199 files) (69.6 K / 716.1 K bytes)
2024-12-09 15:57:31,419 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-whitelightstep_0001.rmap      777 bytes  (13 / 199 files) (70.8 K / 716.1 K bytes)
2024-12-09 15:57:31,467 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-spec2pipeline_0013.rmap    2.1 K bytes  (14 / 199 files) (71.5 K / 716.1 K bytes)
2024-12-09 15:57:31,516 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-resamplespecstep_0002.rmap      709 bytes  (15 / 199 files) (73.6 K / 716.1 K bytes)
2024-12-09 15:57:31,571 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-outlierdetectionstep_0002.rmap      852 bytes  (16 / 199 files) (74.4 K / 716.1 K bytes)
2024-12-09 15:57:31,621 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-jumpstep_0005.rmap      810 bytes  (17 / 199 files) (75.2 K / 716.1 K bytes)
2024-12-09 15:57:31,672 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-image2pipeline_0008.rmap    1.0 K bytes  (18 / 199 files) (76.0 K / 716.1 K bytes)
2024-12-09 15:57:31,728 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-detector1pipeline_0003.rmap    1.1 K bytes  (19 / 199 files) (77.0 K / 716.1 K bytes)
2024-12-09 15:57:31,772 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkpipeline_0003.rmap      872 bytes  (20 / 199 files) (78.1 K / 716.1 K bytes)
2024-12-09 15:57:31,822 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_pars-darkcurrentstep_0001.rmap      622 bytes  (21 / 199 files) (78.9 K / 716.1 K bytes)
2024-12-09 15:57:31,876 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_ote_0030.rmap    1.3 K bytes  (22 / 199 files) (79.6 K / 716.1 K bytes)
2024-12-09 15:57:31,953 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_msaoper_0014.rmap    1.4 K bytes  (23 / 199 files) (80.8 K / 716.1 K bytes)
2024-12-09 15:57:32,010 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_msa_0027.rmap    1.3 K bytes  (24 / 199 files) (82.3 K / 716.1 K bytes)
2024-12-09 15:57:32,061 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_mask_0039.rmap    2.7 K bytes  (25 / 199 files) (83.5 K / 716.1 K bytes)
2024-12-09 15:57:32,148 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_linearity_0017.rmap    1.6 K bytes  (26 / 199 files) (86.2 K / 716.1 K bytes)
2024-12-09 15:57:32,208 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_ipc_0006.rmap      876 bytes  (27 / 199 files) (87.8 K / 716.1 K bytes)
2024-12-09 15:57:32,256 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_ifuslicer_0017.rmap    1.5 K bytes  (28 / 199 files) (88.6 K / 716.1 K bytes)
2024-12-09 15:57:32,307 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_ifupost_0019.rmap    1.5 K bytes  (29 / 199 files) (90.2 K / 716.1 K bytes)
2024-12-09 15:57:32,379 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_ifufore_0017.rmap    1.5 K bytes  (30 / 199 files) (91.7 K / 716.1 K bytes)
2024-12-09 15:57:32,521 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_gain_0023.rmap    1.8 K bytes  (31 / 199 files) (93.2 K / 716.1 K bytes)
2024-12-09 15:57:32,566 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_fpa_0028.rmap    1.3 K bytes  (32 / 199 files) (94.9 K / 716.1 K bytes)
2024-12-09 15:57:32,620 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_fore_0026.rmap    5.0 K bytes  (33 / 199 files) (96.2 K / 716.1 K bytes)
2024-12-09 15:57:32,667 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_flat_0015.rmap    3.8 K bytes  (34 / 199 files) (101.1 K / 716.1 K bytes)
2024-12-09 15:57:32,715 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_fflat_0024.rmap    7.2 K bytes  (35 / 199 files) (105.0 K / 716.1 K bytes)
2024-12-09 15:57:32,764 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_extract1d_0018.rmap    2.3 K bytes  (36 / 199 files) (112.2 K / 716.1 K bytes)
2024-12-09 15:57:32,835 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_disperser_0028.rmap    5.7 K bytes  (37 / 199 files) (114.5 K / 716.1 K bytes)
2024-12-09 15:57:32,886 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_dflat_0007.rmap    1.1 K bytes  (38 / 199 files) (120.2 K / 716.1 K bytes)
2024-12-09 15:57:32,932 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_dark_0069.rmap   32.6 K bytes  (39 / 199 files) (121.3 K / 716.1 K bytes)
2024-12-09 15:57:32,992 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_cubepar_0015.rmap      966 bytes  (40 / 199 files) (153.9 K / 716.1 K bytes)
2024-12-09 15:57:33,038 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_collimator_0026.rmap    1.3 K bytes  (41 / 199 files) (154.9 K / 716.1 K bytes)
2024-12-09 15:57:33,096 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_camera_0026.rmap    1.3 K bytes  (42 / 199 files) (156.2 K / 716.1 K bytes)
2024-12-09 15:57:33,140 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_barshadow_0007.rmap    1.8 K bytes  (43 / 199 files) (157.5 K / 716.1 K bytes)
2024-12-09 15:57:33,279 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_area_0018.rmap    6.3 K bytes  (44 / 199 files) (159.3 K / 716.1 K bytes)
2024-12-09 15:57:33,328 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_apcorr_0009.rmap    5.6 K bytes  (45 / 199 files) (165.5 K / 716.1 K bytes)
2024-12-09 15:57:33,375 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nirspec_0379.imap     5.6 K bytes  (46 / 199 files) (171.1 K / 716.1 K bytes)
2024-12-09 15:57:33,425 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_wfssbkg_0007.rmap    3.1 K bytes  (47 / 199 files) (176.7 K / 716.1 K bytes)
2024-12-09 15:57:33,486 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_wavemap_0008.rmap    2.2 K bytes  (48 / 199 files) (179.8 K / 716.1 K bytes)
2024-12-09 15:57:33,536 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_wavelengthrange_0006.rmap      862 bytes  (49 / 199 files) (182.0 K / 716.1 K bytes)
2024-12-09 15:57:33,586 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_trappars_0004.rmap      753 bytes  (50 / 199 files) (182.9 K / 716.1 K bytes)
2024-12-09 15:57:33,632 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_trapdensity_0005.rmap      705 bytes  (51 / 199 files) (183.7 K / 716.1 K bytes)
2024-12-09 15:57:33,684 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_throughput_0005.rmap    1.3 K bytes  (52 / 199 files) (184.4 K / 716.1 K bytes)
2024-12-09 15:57:33,735 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_superbias_0028.rmap    6.5 K bytes  (53 / 199 files) (185.6 K / 716.1 K bytes)
2024-12-09 15:57:33,783 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_specwcs_0014.rmap    3.1 K bytes  (54 / 199 files) (192.1 K / 716.1 K bytes)
2024-12-09 15:57:33,833 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_spectrace_0008.rmap    2.3 K bytes  (55 / 199 files) (195.2 K / 716.1 K bytes)
2024-12-09 15:57:33,890 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_specprofile_0008.rmap    2.4 K bytes  (56 / 199 files) (197.6 K / 716.1 K bytes)
2024-12-09 15:57:33,935 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_speckernel_0006.rmap    1.0 K bytes  (57 / 199 files) (199.9 K / 716.1 K bytes)
2024-12-09 15:57:33,979 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_saturation_0015.rmap      829 bytes  (58 / 199 files) (201.0 K / 716.1 K bytes)
2024-12-09 15:57:34,023 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_readnoise_0011.rmap      987 bytes  (59 / 199 files) (201.8 K / 716.1 K bytes)
2024-12-09 15:57:34,067 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_photom_0035.rmap    1.3 K bytes  (60 / 199 files) (202.8 K / 716.1 K bytes)
2024-12-09 15:57:34,112 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_persat_0007.rmap      674 bytes  (61 / 199 files) (204.0 K / 716.1 K bytes)
2024-12-09 15:57:34,157 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pathloss_0003.rmap      758 bytes  (62 / 199 files) (204.7 K / 716.1 K bytes)
2024-12-09 15:57:34,204 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-undersamplecorrectionstep_0001.rmap      904 bytes  (63 / 199 files) (205.5 K / 716.1 K bytes)
2024-12-09 15:57:34,255 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-tweakregstep_0012.rmap    3.1 K bytes  (64 / 199 files) (206.4 K / 716.1 K bytes)
2024-12-09 15:57:34,302 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-spec2pipeline_0008.rmap      984 bytes  (65 / 199 files) (209.5 K / 716.1 K bytes)
2024-12-09 15:57:34,355 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-sourcecatalogstep_0002.rmap    2.3 K bytes  (66 / 199 files) (210.5 K / 716.1 K bytes)
2024-12-09 15:57:34,406 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-resamplestep_0002.rmap      687 bytes  (67 / 199 files) (212.8 K / 716.1 K bytes)
2024-12-09 15:57:34,449 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-outlierdetectionstep_0004.rmap    2.7 K bytes  (68 / 199 files) (213.5 K / 716.1 K bytes)
2024-12-09 15:57:34,495 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-jumpstep_0007.rmap    6.4 K bytes  (69 / 199 files) (216.1 K / 716.1 K bytes)
2024-12-09 15:57:34,544 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-image2pipeline_0005.rmap    1.0 K bytes  (70 / 199 files) (222.5 K / 716.1 K bytes)
2024-12-09 15:57:34,592 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-detector1pipeline_0002.rmap    1.0 K bytes  (71 / 199 files) (223.5 K / 716.1 K bytes)
2024-12-09 15:57:34,733 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkpipeline_0002.rmap      868 bytes  (72 / 199 files) (224.6 K / 716.1 K bytes)
2024-12-09 15:57:34,778 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-darkcurrentstep_0001.rmap      591 bytes  (73 / 199 files) (225.4 K / 716.1 K bytes)
2024-12-09 15:57:34,826 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_pars-chargemigrationstep_0004.rmap    5.7 K bytes  (74 / 199 files) (226.0 K / 716.1 K bytes)
2024-12-09 15:57:34,873 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_nrm_0002.rmap      663 bytes  (75 / 199 files) (231.7 K / 716.1 K bytes)
2024-12-09 15:57:34,925 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_mask_0020.rmap      859 bytes  (76 / 199 files) (232.3 K / 716.1 K bytes)
2024-12-09 15:57:34,971 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_linearity_0022.rmap      961 bytes  (77 / 199 files) (233.2 K / 716.1 K bytes)
2024-12-09 15:57:35,019 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_ipc_0007.rmap      651 bytes  (78 / 199 files) (234.2 K / 716.1 K bytes)
2024-12-09 15:57:35,068 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_gain_0011.rmap      797 bytes  (79 / 199 files) (234.8 K / 716.1 K bytes)
2024-12-09 15:57:35,119 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_flat_0023.rmap    5.9 K bytes  (80 / 199 files) (235.6 K / 716.1 K bytes)
2024-12-09 15:57:35,164 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_filteroffset_0010.rmap      853 bytes  (81 / 199 files) (241.5 K / 716.1 K bytes)
2024-12-09 15:57:35,215 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_extract1d_0007.rmap      905 bytes  (82 / 199 files) (242.3 K / 716.1 K bytes)
2024-12-09 15:57:35,266 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_drizpars_0004.rmap      519 bytes  (83 / 199 files) (243.2 K / 716.1 K bytes)
2024-12-09 15:57:35,310 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_distortion_0025.rmap    3.4 K bytes  (84 / 199 files) (243.8 K / 716.1 K bytes)
2024-12-09 15:57:35,357 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_dark_0031.rmap    6.8 K bytes  (85 / 199 files) (247.2 K / 716.1 K bytes)
2024-12-09 15:57:35,412 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_area_0014.rmap    2.7 K bytes  (86 / 199 files) (254.0 K / 716.1 K bytes)
2024-12-09 15:57:35,455 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_apcorr_0010.rmap    4.3 K bytes  (87 / 199 files) (256.7 K / 716.1 K bytes)
2024-12-09 15:57:35,502 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_abvegaoffset_0004.rmap    1.4 K bytes  (88 / 199 files) (261.0 K / 716.1 K bytes)
2024-12-09 15:57:35,566 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_niriss_0256.imap      5.6 K bytes  (89 / 199 files) (262.3 K / 716.1 K bytes)
2024-12-09 15:57:35,615 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_wfssbkg_0004.rmap    7.2 K bytes  (90 / 199 files) (268.0 K / 716.1 K bytes)
2024-12-09 15:57:35,658 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_wavelengthrange_0010.rmap      996 bytes  (91 / 199 files) (275.2 K / 716.1 K bytes)
2024-12-09 15:57:35,717 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_tsophot_0003.rmap      896 bytes  (92 / 199 files) (276.2 K / 716.1 K bytes)
2024-12-09 15:57:35,762 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_trappars_0003.rmap    1.6 K bytes  (93 / 199 files) (277.0 K / 716.1 K bytes)
2024-12-09 15:57:35,824 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_trapdensity_0003.rmap    1.6 K bytes  (94 / 199 files) (278.6 K / 716.1 K bytes)
2024-12-09 15:57:35,873 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_superbias_0017.rmap   16.1 K bytes  (95 / 199 files) (280.3 K / 716.1 K bytes)
2024-12-09 15:57:35,936 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_specwcs_0022.rmap    7.1 K bytes  (96 / 199 files) (296.3 K / 716.1 K bytes)
2024-12-09 15:57:35,983 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_saturation_0010.rmap    2.2 K bytes  (97 / 199 files) (303.4 K / 716.1 K bytes)
2024-12-09 15:57:36,035 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_readnoise_0024.rmap   23.1 K bytes  (98 / 199 files) (305.6 K / 716.1 K bytes)
2024-12-09 15:57:36,089 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_psfmask_0008.rmap   28.4 K bytes  (99 / 199 files) (328.7 K / 716.1 K bytes)
2024-12-09 15:57:36,145 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_photom_0028.rmap    3.4 K bytes  (100 / 199 files) (357.0 K / 716.1 K bytes)
2024-12-09 15:57:36,233 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_persat_0005.rmap    1.6 K bytes  (101 / 199 files) (360.4 K / 716.1 K bytes)
2024-12-09 15:57:36,277 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-whitelightstep_0003.rmap    1.5 K bytes  (102 / 199 files) (361.9 K / 716.1 K bytes)
2024-12-09 15:57:36,323 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-tweakregstep_0003.rmap    4.5 K bytes  (103 / 199 files) (363.4 K / 716.1 K bytes)
2024-12-09 15:57:36,372 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-spec2pipeline_0008.rmap      984 bytes  (104 / 199 files) (367.9 K / 716.1 K bytes)
2024-12-09 15:57:36,420 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-sourcecatalogstep_0002.rmap    4.6 K bytes  (105 / 199 files) (368.9 K / 716.1 K bytes)
2024-12-09 15:57:36,471 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-resamplestep_0002.rmap      687 bytes  (106 / 199 files) (373.5 K / 716.1 K bytes)
2024-12-09 15:57:36,553 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-outlierdetectionstep_0003.rmap      940 bytes  (107 / 199 files) (374.2 K / 716.1 K bytes)
2024-12-09 15:57:36,602 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-jumpstep_0005.rmap      806 bytes  (108 / 199 files) (375.1 K / 716.1 K bytes)
2024-12-09 15:57:36,652 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-image2pipeline_0003.rmap    1.0 K bytes  (109 / 199 files) (375.9 K / 716.1 K bytes)
2024-12-09 15:57:36,695 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-detector1pipeline_0003.rmap    1.0 K bytes  (110 / 199 files) (377.0 K / 716.1 K bytes)
2024-12-09 15:57:36,750 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkpipeline_0002.rmap      868 bytes  (111 / 199 files) (378.0 K / 716.1 K bytes)
2024-12-09 15:57:36,802 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_pars-darkcurrentstep_0001.rmap      618 bytes  (112 / 199 files) (378.9 K / 716.1 K bytes)
2024-12-09 15:57:36,850 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_mask_0011.rmap    3.5 K bytes  (113 / 199 files) (379.5 K / 716.1 K bytes)
2024-12-09 15:57:36,911 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_linearity_0011.rmap    2.4 K bytes  (114 / 199 files) (383.0 K / 716.1 K bytes)
2024-12-09 15:57:36,958 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_ipc_0003.rmap    2.0 K bytes  (115 / 199 files) (385.4 K / 716.1 K bytes)
2024-12-09 15:57:37,003 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_gain_0016.rmap    2.1 K bytes  (116 / 199 files) (387.4 K / 716.1 K bytes)
2024-12-09 15:57:37,048 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_flat_0027.rmap   51.7 K bytes  (117 / 199 files) (389.5 K / 716.1 K bytes)
2024-12-09 15:57:37,109 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_filteroffset_0004.rmap    1.4 K bytes  (118 / 199 files) (441.2 K / 716.1 K bytes)
2024-12-09 15:57:37,165 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_extract1d_0004.rmap      842 bytes  (119 / 199 files) (442.6 K / 716.1 K bytes)
2024-12-09 15:57:37,212 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_drizpars_0001.rmap      519 bytes  (120 / 199 files) (443.4 K / 716.1 K bytes)
2024-12-09 15:57:37,263 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_distortion_0033.rmap   53.4 K bytes  (121 / 199 files) (444.0 K / 716.1 K bytes)
2024-12-09 15:57:37,320 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_dark_0044.rmap   26.3 K bytes  (122 / 199 files) (497.3 K / 716.1 K bytes)
2024-12-09 15:57:37,375 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_area_0012.rmap   33.5 K bytes  (123 / 199 files) (523.6 K / 716.1 K bytes)
2024-12-09 15:57:37,432 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_apcorr_0008.rmap    4.3 K bytes  (124 / 199 files) (557.2 K / 716.1 K bytes)
2024-12-09 15:57:37,478 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_abvegaoffset_0003.rmap    1.3 K bytes  (125 / 199 files) (561.4 K / 716.1 K bytes)
2024-12-09 15:57:37,526 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_nircam_0294.imap      5.4 K bytes  (126 / 199 files) (562.7 K / 716.1 K bytes)
2024-12-09 15:57:37,574 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_wavelengthrange_0027.rmap      929 bytes  (127 / 199 files) (568.1 K / 716.1 K bytes)
2024-12-09 15:57:37,626 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_tsophot_0004.rmap      882 bytes  (128 / 199 files) (569.1 K / 716.1 K bytes)
2024-12-09 15:57:37,673 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_straymask_0009.rmap      987 bytes  (129 / 199 files) (570.0 K / 716.1 K bytes)
2024-12-09 15:57:37,726 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_specwcs_0042.rmap    5.8 K bytes  (130 / 199 files) (570.9 K / 716.1 K bytes)
2024-12-09 15:57:37,768 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_saturation_0015.rmap    1.2 K bytes  (131 / 199 files) (576.7 K / 716.1 K bytes)
2024-12-09 15:57:37,815 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_rscd_0008.rmap    1.0 K bytes  (132 / 199 files) (577.9 K / 716.1 K bytes)
2024-12-09 15:57:37,861 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_resol_0006.rmap      790 bytes  (133 / 199 files) (578.9 K / 716.1 K bytes)
2024-12-09 15:57:37,908 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_reset_0026.rmap    3.9 K bytes  (134 / 199 files) (579.7 K / 716.1 K bytes)
2024-12-09 15:57:37,957 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_regions_0033.rmap    5.2 K bytes  (135 / 199 files) (583.6 K / 716.1 K bytes)
2024-12-09 15:57:38,003 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_readnoise_0023.rmap    1.6 K bytes  (136 / 199 files) (588.8 K / 716.1 K bytes)
2024-12-09 15:57:38,054 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_psfmask_0009.rmap    2.1 K bytes  (137 / 199 files) (590.4 K / 716.1 K bytes)
2024-12-09 15:57:38,100 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_photom_0056.rmap    3.7 K bytes  (138 / 199 files) (592.6 K / 716.1 K bytes)
2024-12-09 15:57:38,145 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pathloss_0005.rmap      866 bytes  (139 / 199 files) (596.3 K / 716.1 K bytes)
2024-12-09 15:57:38,194 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-whitelightstep_0003.rmap      912 bytes  (140 / 199 files) (597.2 K / 716.1 K bytes)
2024-12-09 15:57:38,245 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-tweakregstep_0003.rmap    1.8 K bytes  (141 / 199 files) (598.1 K / 716.1 K bytes)
2024-12-09 15:57:38,298 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-spec3pipeline_0008.rmap      816 bytes  (142 / 199 files) (599.9 K / 716.1 K bytes)
2024-12-09 15:57:38,342 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-spec2pipeline_0012.rmap    1.3 K bytes  (143 / 199 files) (600.7 K / 716.1 K bytes)
2024-12-09 15:57:38,387 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-sourcecatalogstep_0003.rmap    1.9 K bytes  (144 / 199 files) (602.1 K / 716.1 K bytes)
2024-12-09 15:57:38,435 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplestep_0002.rmap      677 bytes  (145 / 199 files) (604.0 K / 716.1 K bytes)
2024-12-09 15:57:38,485 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-resamplespecstep_0002.rmap      706 bytes  (146 / 199 files) (604.6 K / 716.1 K bytes)
2024-12-09 15:57:38,538 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-outlierdetectionstep_0017.rmap    3.4 K bytes  (147 / 199 files) (605.3 K / 716.1 K bytes)
2024-12-09 15:57:38,587 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-jumpstep_0009.rmap    1.4 K bytes  (148 / 199 files) (608.7 K / 716.1 K bytes)
2024-12-09 15:57:38,629 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-image2pipeline_0007.rmap      983 bytes  (149 / 199 files) (610.2 K / 716.1 K bytes)
2024-12-09 15:57:38,678 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-extract1dstep_0002.rmap      728 bytes  (150 / 199 files) (611.1 K / 716.1 K bytes)
2024-12-09 15:57:38,721 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-emicorrstep_0002.rmap      796 bytes  (151 / 199 files) (611.9 K / 716.1 K bytes)
2024-12-09 15:57:38,772 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-detector1pipeline_0009.rmap    1.4 K bytes  (152 / 199 files) (612.7 K / 716.1 K bytes)
2024-12-09 15:57:38,816 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-darkpipeline_0002.rmap      860 bytes  (153 / 199 files) (614.1 K / 716.1 K bytes)
2024-12-09 15:57:38,860 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_pars-darkcurrentstep_0002.rmap      683 bytes  (154 / 199 files) (614.9 K / 716.1 K bytes)
2024-12-09 15:57:38,911 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_mrsxartcorr_0002.rmap    2.2 K bytes  (155 / 199 files) (615.6 K / 716.1 K bytes)
2024-12-09 15:57:38,957 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_mrsptcorr_0005.rmap    2.0 K bytes  (156 / 199 files) (617.8 K / 716.1 K bytes)
2024-12-09 15:57:39,003 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_mask_0023.rmap    3.5 K bytes  (157 / 199 files) (619.7 K / 716.1 K bytes)
2024-12-09 15:57:39,048 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_linearity_0018.rmap    2.8 K bytes  (158 / 199 files) (623.2 K / 716.1 K bytes)
2024-12-09 15:57:39,111 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_ipc_0008.rmap      700 bytes  (159 / 199 files) (626.1 K / 716.1 K bytes)
2024-12-09 15:57:39,155 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_gain_0013.rmap    3.9 K bytes  (160 / 199 files) (626.8 K / 716.1 K bytes)
2024-12-09 15:57:39,204 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_fringefreq_0003.rmap    1.4 K bytes  (161 / 199 files) (630.7 K / 716.1 K bytes)
2024-12-09 15:57:39,255 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_fringe_0019.rmap    3.9 K bytes  (162 / 199 files) (632.1 K / 716.1 K bytes)
2024-12-09 15:57:39,309 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_flat_0065.rmap   15.5 K bytes  (163 / 199 files) (636.0 K / 716.1 K bytes)
2024-12-09 15:57:39,367 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_filteroffset_0025.rmap    2.5 K bytes  (164 / 199 files) (651.5 K / 716.1 K bytes)
2024-12-09 15:57:39,416 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_extract1d_0020.rmap    1.4 K bytes  (165 / 199 files) (654.0 K / 716.1 K bytes)
2024-12-09 15:57:39,475 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_emicorr_0003.rmap      663 bytes  (166 / 199 files) (655.4 K / 716.1 K bytes)
2024-12-09 15:57:39,519 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_drizpars_0002.rmap      511 bytes  (167 / 199 files) (656.0 K / 716.1 K bytes)
2024-12-09 15:57:39,574 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_distortion_0040.rmap    4.9 K bytes  (168 / 199 files) (656.5 K / 716.1 K bytes)
2024-12-09 15:57:39,620 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_dark_0036.rmap    4.4 K bytes  (169 / 199 files) (661.5 K / 716.1 K bytes)
2024-12-09 15:57:39,667 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_cubepar_0017.rmap      800 bytes  (170 / 199 files) (665.8 K / 716.1 K bytes)
2024-12-09 15:57:39,713 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_area_0015.rmap      866 bytes  (171 / 199 files) (666.6 K / 716.1 K bytes)
2024-12-09 15:57:39,764 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_apcorr_0019.rmap    5.0 K bytes  (172 / 199 files) (667.5 K / 716.1 K bytes)
2024-12-09 15:57:39,814 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_abvegaoffset_0002.rmap    1.3 K bytes  (173 / 199 files) (672.4 K / 716.1 K bytes)
2024-12-09 15:57:39,874 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_miri_0415.imap        5.7 K bytes  (174 / 199 files) (673.7 K / 716.1 K bytes)
2024-12-09 15:57:39,918 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_trappars_0004.rmap      903 bytes  (175 / 199 files) (679.4 K / 716.1 K bytes)
2024-12-09 15:57:39,964 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_trapdensity_0006.rmap      930 bytes  (176 / 199 files) (680.3 K / 716.1 K bytes)
2024-12-09 15:57:40,018 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_superbias_0017.rmap    3.8 K bytes  (177 / 199 files) (681.2 K / 716.1 K bytes)
2024-12-09 15:57:40,065 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_saturation_0009.rmap      779 bytes  (178 / 199 files) (685.0 K / 716.1 K bytes)
2024-12-09 15:57:40,119 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_readnoise_0011.rmap    1.3 K bytes  (179 / 199 files) (685.8 K / 716.1 K bytes)
2024-12-09 15:57:40,167 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_photom_0014.rmap    1.1 K bytes  (180 / 199 files) (687.0 K / 716.1 K bytes)
2024-12-09 15:57:40,216 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_persat_0006.rmap      884 bytes  (181 / 199 files) (688.2 K / 716.1 K bytes)
2024-12-09 15:57:40,262 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-tweakregstep_0002.rmap      850 bytes  (182 / 199 files) (689.1 K / 716.1 K bytes)
2024-12-09 15:57:40,314 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-sourcecatalogstep_0001.rmap      636 bytes  (183 / 199 files) (689.9 K / 716.1 K bytes)
2024-12-09 15:57:40,358 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-outlierdetectionstep_0001.rmap      654 bytes  (184 / 199 files) (690.5 K / 716.1 K bytes)
2024-12-09 15:57:40,401 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-image2pipeline_0005.rmap      974 bytes  (185 / 199 files) (691.2 K / 716.1 K bytes)
2024-12-09 15:57:40,452 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-detector1pipeline_0002.rmap    1.0 K bytes  (186 / 199 files) (692.2 K / 716.1 K bytes)
2024-12-09 15:57:40,507 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_pars-darkpipeline_0002.rmap      856 bytes  (187 / 199 files) (693.2 K / 716.1 K bytes)
2024-12-09 15:57:40,553 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_mask_0023.rmap    1.1 K bytes  (188 / 199 files) (694.1 K / 716.1 K bytes)
2024-12-09 15:57:40,601 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_linearity_0015.rmap      925 bytes  (189 / 199 files) (695.1 K / 716.1 K bytes)
2024-12-09 15:57:40,644 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_ipc_0003.rmap       614 bytes  (190 / 199 files) (696.0 K / 716.1 K bytes)
2024-12-09 15:57:40,693 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_gain_0010.rmap      890 bytes  (191 / 199 files) (696.6 K / 716.1 K bytes)
2024-12-09 15:57:40,739 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_flat_0009.rmap    1.1 K bytes  (192 / 199 files) (697.5 K / 716.1 K bytes)
2024-12-09 15:57:40,787 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_distortion_0011.rmap    1.2 K bytes  (193 / 199 files) (698.7 K / 716.1 K bytes)
2024-12-09 15:57:40,831 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_dark_0017.rmap    4.3 K bytes  (194 / 199 files) (699.9 K / 716.1 K bytes)
2024-12-09 15:57:40,879 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_area_0010.rmap    1.2 K bytes  (195 / 199 files) (704.2 K / 716.1 K bytes)
2024-12-09 15:57:40,923 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_apcorr_0004.rmap    4.0 K bytes  (196 / 199 files) (705.3 K / 716.1 K bytes)
2024-12-09 15:57:40,965 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_abvegaoffset_0002.rmap    1.3 K bytes  (197 / 199 files) (709.3 K / 716.1 K bytes)
2024-12-09 15:57:41,012 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_fgs_0116.imap         5.0 K bytes  (198 / 199 files) (710.6 K / 716.1 K bytes)
2024-12-09 15:57:41,061 - CRDS - INFO -  Fetching  /home/runner/crds/mappings/jwst/jwst_1293.pmap               580 bytes  (199 / 199 files) (715.5 K / 716.1 K bytes)
2024-12-09 15:57:41,586 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 15:57:41,590 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf    1.8 K bytes  (1 / 1 files) (0 / 1.8 K bytes)
2024-12-09 15:57:41,643 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 15:57:41,657 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf    1.7 K bytes  (1 / 1 files) (0 / 1.7 K bytes)
2024-12-09 15:57:41,704 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 15:57:41,724 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 15:57:41,725 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 15:57:41,726 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 15:57:41,727 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 15:57:41,728 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 15:57:41,729 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 15:57:41,730 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 15:57:41,731 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 15:57:41,733 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 15:57:41,734 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 15:57:41,735 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 15:57:41,736 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 15:57:41,736 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 15:57:41,737 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 15:57:41,738 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 15:57:41,739 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 15:57:41,741 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 15:57:41,742 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 15:57:41,743 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 15:57:41,934 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca2_uncal.fits',).
2024-12-09 15:57:41,955 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 15:57:42,051 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 15:57:42,056 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits    6.3 G bytes  (1 / 7 files) (0 / 6.6 G bytes)
2024-12-09 16:00:25,801 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits   16.8 M bytes  (2 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 16:00:26,038 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits  151.0 M bytes  (3 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 16:00:28,104 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits   16.8 M bytes  (4 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:00:28,405 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits   16.8 M bytes  (5 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:00:28,702 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits   33.6 M bytes  (6 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:00:29,343 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits   50.4 M bytes  (7 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:00:30,164 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits'.
2024-12-09 16:00:30,165 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits'.
2024-12-09 16:00:30,165 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits'.
2024-12-09 16:00:30,166 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits'.
2024-12-09 16:00:30,167 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits'.
2024-12-09 16:00:30,168 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:00:30,168 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:00:30,169 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:00:30,169 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits'.
2024-12-09 16:00:30,170 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits'.
2024-12-09 16:00:30,171 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:00:30,525 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:30,593 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:00:30,593 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:00:30,595 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:00:30,767 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:30,787 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits
2024-12-09 16:00:31,016 - CRDS - INFO -  Calibration SW Found: jwst 1.15.1 (/opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst-1.15.1.dist-info)
2024-12-09 16:00:31,077 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:00:31,253 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:31,274 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits
2024-12-09 16:00:31,329 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:31,383 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:32,435 - stpipe.Detector1Pipeline.saturation - INFO - Detected 8870 saturated pixels
2024-12-09 16:00:32,454 - stpipe.Detector1Pipeline.saturation - INFO - Detected 1 A/D floor pixels
2024-12-09 16:00:32,462 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:00:32,626 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:32,627 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:00:32,797 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:32,817 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits
2024-12-09 16:00:33,011 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:00:33,192 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:33,260 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:00:33,261 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:00:33,261 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:00:33,261 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:00:33,262 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:00:33,262 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:00:33,263 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:00:33,263 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:00:34,886 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:00:35,038 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:35,058 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits
2024-12-09 16:00:35,128 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,129 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,130 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,130 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,131 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,132 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,132 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:35,634 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:00:35,787 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:35,787 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:00:35,936 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:35,956 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits
2024-12-09 16:00:37,356 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:37,401 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:00:37,423 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:00:37,424 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 16:00:38,085 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:00:38,281 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:38,282 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:00:38,481 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:38,493 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:00:38,493 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:00:38,505 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 16:00:38,530 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 16:00:38,753 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:00:38,786 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:00:41,474 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:00:44,796 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 63
2024-12-09 16:00:44,796 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 6.04252 sec
2024-12-09 16:00:44,850 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 6.356841
2024-12-09 16:00:44,852 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:00:45,000 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:45,027 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 16:00:45,028 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 16:00:45,056 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:00:45,056 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:00:45,234 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:00:47,820 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.5811450481414795
2024-12-09 16:00:48,039 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:00:48,185 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:48,212 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:00:48,213 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:00:48,215 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:00:48,350 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrca2_uncal.fits>,).
2024-12-09 16:00:48,379 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:00:48,380 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:00:48,382 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:00:48,536 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rateints.fits
2024-12-09 16:00:48,537 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:00:48,538 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:00:48,683 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits
2024-12-09 16:00:48,684 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 16:00:48,767 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 16:00:48,771 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 16:00:48,783 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 16:00:48,799 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 16:00:48,800 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 16:00:48,801 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 16:00:48,802 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 16:00:48,803 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 16:00:48,804 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 16:00:48,805 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 16:00:48,806 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 16:00:48,807 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 16:00:48,807 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 16:00:48,808 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 16:00:48,809 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 16:00:48,810 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 16:00:48,811 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 16:00:48,812 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 16:00:48,812 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 16:00:48,815 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 16:00:48,816 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 16:00:48,817 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 16:00:48,994 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrca4_uncal.fits',).
2024-12-09 16:00:49,013 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 16:00:49,085 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca4_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 16:00:49,089 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits    6.3 G bytes  (1 / 7 files) (0 / 6.6 G bytes)
2024-12-09 16:02:55,146 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits   16.8 M bytes  (2 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 16:02:55,459 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits  151.0 M bytes  (3 / 7 files) (6.3 G / 6.6 G bytes)
2024-12-09 16:02:56,743 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits   16.8 M bytes  (4 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:02:56,968 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits   16.8 M bytes  (5 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:02:57,255 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits   33.6 M bytes  (6 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:02:57,627 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits   50.4 M bytes  (7 / 7 files) (6.5 G / 6.6 G bytes)
2024-12-09 16:02:58,140 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits'.
2024-12-09 16:02:58,141 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits'.
2024-12-09 16:02:58,142 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits'.
2024-12-09 16:02:58,142 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits'.
2024-12-09 16:02:58,142 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits'.
2024-12-09 16:02:58,143 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:02:58,143 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:02:58,144 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:02:58,144 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits'.
2024-12-09 16:02:58,144 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits'.
2024-12-09 16:02:58,145 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:02:58,465 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:02:58,530 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:02:58,530 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:02:58,532 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:02:58,711 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:02:58,732 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits
2024-12-09 16:02:58,959 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:02:59,128 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:02:59,148 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits
2024-12-09 16:02:59,194 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:02:59,247 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:00,172 - stpipe.Detector1Pipeline.saturation - INFO - Detected 5208 saturated pixels
2024-12-09 16:03:00,186 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 16:03:00,193 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:03:00,336 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:00,337 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:03:00,465 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:00,484 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits
2024-12-09 16:03:00,659 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:03:00,807 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:00,863 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:03:00,864 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:03:00,864 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:03:00,864 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:03:00,865 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:03:00,865 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:03:00,866 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:03:00,866 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:03:02,468 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:03:02,612 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:02,630 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits
2024-12-09 16:03:02,694 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,695 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,696 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,696 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,697 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,697 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:02,698 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:03,149 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:03:03,319 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:03,320 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:03:03,484 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:03,503 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits
2024-12-09 16:03:04,791 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:04,832 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:03:04,844 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:03:04,844 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 16:03:05,447 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:03:05,588 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:05,589 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:03:05,719 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:05,728 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:03:05,728 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:03:05,738 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 16:03:05,761 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 16:03:05,948 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:03:05,973 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:03:08,495 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:03:11,037 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 59
2024-12-09 16:03:11,037 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.08876 sec
2024-12-09 16:03:11,100 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.371959
2024-12-09 16:03:11,102 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:03:11,262 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:11,292 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 16:03:11,293 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 16:03:11,322 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:03:11,323 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:03:11,516 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:03:14,160 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.63678240776062
2024-12-09 16:03:14,385 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:03:14,557 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:14,588 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:03:14,588 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:03:14,590 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:03:14,772 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrca4_uncal.fits>,).
2024-12-09 16:03:14,805 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:03:14,806 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:03:14,807 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:03:14,973 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rateints.fits
2024-12-09 16:03:14,974 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:03:14,975 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:03:15,138 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits
2024-12-09 16:03:15,139 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 16:03:15,228 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 16:03:15,232 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 16:03:15,244 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 16:03:15,261 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 16:03:15,262 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 16:03:15,263 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 16:03:15,264 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 16:03:15,266 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 16:03:15,267 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 16:03:15,268 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 16:03:15,269 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 16:03:15,270 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 16:03:15,271 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 16:03:15,273 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 16:03:15,274 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 16:03:15,275 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 16:03:15,276 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 16:03:15,277 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 16:03:15,278 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 16:03:15,281 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 16:03:15,282 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 16:03:15,283 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 16:03:15,496 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00001_nrcalong_uncal.fits',).
2024-12-09 16:03:15,516 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 16:03:15,590 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 16:03:15,595 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits    4.4 G bytes  (1 / 7 files) (0 / 4.7 G bytes)
2024-12-09 16:04:33,205 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits   16.8 M bytes  (2 / 7 files) (4.4 G / 4.7 G bytes)
2024-12-09 16:04:33,580 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits  151.0 M bytes  (3 / 7 files) (4.4 G / 4.7 G bytes)
2024-12-09 16:04:37,291 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits   16.8 M bytes  (4 / 7 files) (4.5 G / 4.7 G bytes)
2024-12-09 16:04:37,572 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits   16.8 M bytes  (5 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 16:04:38,497 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits   33.6 M bytes  (6 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 16:04:39,197 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits   50.4 M bytes  (7 / 7 files) (4.6 G / 4.7 G bytes)
2024-12-09 16:04:41,451 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits'.
2024-12-09 16:04:41,452 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2024-12-09 16:04:41,452 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2024-12-09 16:04:41,453 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2024-12-09 16:04:41,453 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits'.
2024-12-09 16:04:41,454 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:04:41,454 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:04:41,454 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:04:41,455 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2024-12-09 16:04:41,455 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits'.
2024-12-09 16:04:41,457 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:04:41,731 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:41,775 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:04:41,776 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:04:41,777 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:04:41,915 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:41,933 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2024-12-09 16:04:42,147 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:04:42,288 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:42,307 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2024-12-09 16:04:42,348 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:42,389 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:43,290 - stpipe.Detector1Pipeline.saturation - INFO - Detected 27972 saturated pixels
2024-12-09 16:04:43,303 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 16:04:43,309 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:04:43,447 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:43,448 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:04:43,578 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:43,596 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits
2024-12-09 16:04:43,778 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:04:43,916 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:43,972 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:04:43,973 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:04:43,973 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:04:43,974 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:04:43,974 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:04:43,975 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:04:43,975 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:04:43,976 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:04:45,575 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:04:45,717 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:45,736 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2024-12-09 16:04:45,803 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,803 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,804 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,804 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,805 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,805 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:45,806 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:04:46,295 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:04:46,447 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:46,447 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:04:46,592 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:46,611 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits
2024-12-09 16:04:48,194 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:04:48,195 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=130, nframes=1, groupgap=0
2024-12-09 16:04:48,688 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:04:48,834 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:48,835 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:04:48,978 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:04:48,988 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:04:48,989 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:04:48,999 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 16:04:49,022 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 16:04:49,238 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:04:49,264 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:04:52,927 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:05:03,843 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 246
2024-12-09 16:05:03,843 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 14.605 sec
2024-12-09 16:05:03,893 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 14.905148
2024-12-09 16:05:03,896 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:05:04,038 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:05:04,066 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 16:05:04,067 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 16:05:04,094 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:05:04,095 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:05:04,362 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:05:06,968 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.6025452613830566
2024-12-09 16:05:07,179 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:05:07,333 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:05:07,366 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:07,367 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:07,369 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:07,516 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00001_nrcalong_uncal.fits>,).
2024-12-09 16:05:07,546 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:07,546 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:07,548 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:07,707 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rateints.fits
2024-12-09 16:05:07,708 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:05:07,709 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:05:07,858 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits
2024-12-09 16:05:07,858 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 16:05:07,938 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 16:05:07,942 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 16:05:07,954 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 16:05:07,971 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 16:05:07,971 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 16:05:07,972 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 16:05:07,974 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 16:05:07,974 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 16:05:07,975 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 16:05:07,976 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 16:05:07,977 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 16:05:07,978 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 16:05:07,979 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 16:05:07,980 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 16:05:07,981 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 16:05:07,981 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 16:05:07,982 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 16:05:07,983 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 16:05:07,985 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 16:05:07,987 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 16:05:07,988 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 16:05:07,988 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 16:05:08,153 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca2_uncal.fits',).
2024-12-09 16:05:08,173 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 16:05:08,246 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca2_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 16:05:08,250 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits'.
2024-12-09 16:05:08,251 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits'.
2024-12-09 16:05:08,251 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits'.
2024-12-09 16:05:08,252 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits'.
2024-12-09 16:05:08,252 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits'.
2024-12-09 16:05:08,253 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:05:08,253 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:05:08,253 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:05:08,254 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits'.
2024-12-09 16:05:08,254 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits'.
2024-12-09 16:05:08,255 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:05:08,532 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:08,565 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:05:08,565 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:05:08,566 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:05:08,720 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:08,738 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0072.fits
2024-12-09 16:05:08,915 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:05:09,054 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:09,073 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0104.fits
2024-12-09 16:05:09,116 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:09,168 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:10,028 - stpipe.Detector1Pipeline.saturation - INFO - Detected 9023 saturated pixels
2024-12-09 16:05:10,042 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 16:05:10,049 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:05:10,220 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:10,221 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:05:10,386 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:10,404 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0186.fits
2024-12-09 16:05:10,570 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:05:10,709 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:10,764 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:05:10,765 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:05:10,765 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:05:10,765 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:05:10,766 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:05:10,766 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:05:10,767 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:05:10,767 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:05:12,351 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:05:12,511 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:12,531 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0051.fits
2024-12-09 16:05:12,592 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,593 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,593 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,594 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,594 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,595 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:12,595 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:13,038 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:05:13,186 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:13,186 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:05:13,327 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:13,346 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0342.fits
2024-12-09 16:05:14,359 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:14,403 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:14,423 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:05:14,423 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 16:05:15,044 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:05:15,200 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:15,200 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:05:15,340 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:15,349 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:05:15,350 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:05:15,360 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 16:05:15,382 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 16:05:15,575 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:05:15,601 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:05:18,279 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:05:21,476 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 60
2024-12-09 16:05:21,476 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.90115 sec
2024-12-09 16:05:21,528 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 6.179746
2024-12-09 16:05:21,531 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:05:21,673 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:21,701 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0224.fits
2024-12-09 16:05:21,701 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0093.fits
2024-12-09 16:05:21,728 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:05:21,728 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:05:21,904 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:05:24,483 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.5758512020111084
2024-12-09 16:05:24,690 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:05:24,831 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:24,858 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:24,859 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:24,860 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:25,001 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrca2_uncal.fits>,).
2024-12-09 16:05:25,030 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:25,030 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:25,032 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:25,182 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rateints.fits
2024-12-09 16:05:25,183 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:05:25,184 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:05:25,329 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits
2024-12-09 16:05:25,329 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 16:05:25,408 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 16:05:25,412 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 16:05:25,424 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 16:05:25,440 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 16:05:25,441 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 16:05:25,442 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 16:05:25,443 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 16:05:25,444 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 16:05:25,445 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 16:05:25,446 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 16:05:25,447 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 16:05:25,448 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 16:05:25,449 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 16:05:25,450 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 16:05:25,450 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 16:05:25,451 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 16:05:25,452 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 16:05:25,453 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 16:05:25,454 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 16:05:25,455 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 16:05:25,456 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 16:05:25,457 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 16:05:25,623 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrca4_uncal.fits',).
2024-12-09 16:05:25,643 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 16:05:25,715 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca4_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 16:05:25,719 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits'.
2024-12-09 16:05:25,720 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits'.
2024-12-09 16:05:25,720 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits'.
2024-12-09 16:05:25,721 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits'.
2024-12-09 16:05:25,721 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits'.
2024-12-09 16:05:25,722 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:05:25,722 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:05:25,722 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:05:25,722 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits'.
2024-12-09 16:05:25,723 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits'.
2024-12-09 16:05:25,724 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:05:26,012 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:26,046 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:05:26,047 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:05:26,048 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:05:26,207 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:26,227 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0078.fits
2024-12-09 16:05:26,407 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:05:26,564 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:26,583 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0099.fits
2024-12-09 16:05:26,627 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:26,680 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:27,568 - stpipe.Detector1Pipeline.saturation - INFO - Detected 4958 saturated pixels
2024-12-09 16:05:27,582 - stpipe.Detector1Pipeline.saturation - INFO - Detected 1 A/D floor pixels
2024-12-09 16:05:27,588 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:05:27,742 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:27,743 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:05:27,874 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:27,893 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0212.fits
2024-12-09 16:05:28,061 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:05:28,206 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:28,262 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:05:28,263 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:05:28,263 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:05:28,264 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:05:28,264 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:05:28,265 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:05:28,265 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:05:28,265 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:05:29,853 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:05:30,027 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:30,046 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0055.fits
2024-12-09 16:05:30,103 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,104 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,105 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,105 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,106 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,106 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,107 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:30,562 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:05:30,732 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:30,733 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:05:30,909 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:30,930 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0334.fits
2024-12-09 16:05:31,973 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:32,011 - stpipe.Detector1Pipeline.dark_current - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:32,022 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:05:32,023 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=187, nframes=1, groupgap=0
2024-12-09 16:05:32,609 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:05:32,767 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:32,767 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:05:32,916 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:32,925 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:05:32,926 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:05:32,937 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 16:05:32,959 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 16:05:33,149 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:05:33,174 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:05:35,690 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:05:38,162 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 83
2024-12-09 16:05:38,163 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 5.01284 sec
2024-12-09 16:05:38,214 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 5.288323
2024-12-09 16:05:38,216 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:05:38,363 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:38,391 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0248.fits
2024-12-09 16:05:38,392 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0096.fits
2024-12-09 16:05:38,419 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:05:38,419 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:05:38,575 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:05:41,184 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.6048922538757324
2024-12-09 16:05:41,393 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:05:41,536 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:41,564 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:41,565 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:41,566 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:41,706 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrca4_uncal.fits>,).
2024-12-09 16:05:41,736 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:05:41,736 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:05:41,738 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:05:41,890 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rateints.fits
2024-12-09 16:05:41,890 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:05:41,891 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:05:42,035 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits
2024-12-09 16:05:42,036 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
2024-12-09 16:05:42,113 - CRDS - ERROR -  Error determining best reference for 'pars-darkcurrentstep'  =   No match found.
2024-12-09 16:05:42,117 - stpipe - INFO - PARS-JUMPSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-jumpstep_0003.asdf
2024-12-09 16:05:42,129 - stpipe - INFO - PARS-DETECTOR1PIPELINE parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-detector1pipeline_0003.asdf
2024-12-09 16:05:42,146 - stpipe.Detector1Pipeline - INFO - Detector1Pipeline instance created.
2024-12-09 16:05:42,147 - stpipe.Detector1Pipeline.group_scale - INFO - GroupScaleStep instance created.
2024-12-09 16:05:42,147 - stpipe.Detector1Pipeline.dq_init - INFO - DQInitStep instance created.
2024-12-09 16:05:42,149 - stpipe.Detector1Pipeline.emicorr - INFO - EmiCorrStep instance created.
2024-12-09 16:05:42,150 - stpipe.Detector1Pipeline.saturation - INFO - SaturationStep instance created.
2024-12-09 16:05:42,151 - stpipe.Detector1Pipeline.ipc - INFO - IPCStep instance created.
2024-12-09 16:05:42,152 - stpipe.Detector1Pipeline.superbias - INFO - SuperBiasStep instance created.
2024-12-09 16:05:42,153 - stpipe.Detector1Pipeline.refpix - INFO - RefPixStep instance created.
2024-12-09 16:05:42,154 - stpipe.Detector1Pipeline.rscd - INFO - RscdStep instance created.
2024-12-09 16:05:42,155 - stpipe.Detector1Pipeline.firstframe - INFO - FirstFrameStep instance created.
2024-12-09 16:05:42,156 - stpipe.Detector1Pipeline.lastframe - INFO - LastFrameStep instance created.
2024-12-09 16:05:42,157 - stpipe.Detector1Pipeline.linearity - INFO - LinearityStep instance created.
2024-12-09 16:05:42,158 - stpipe.Detector1Pipeline.dark_current - INFO - DarkCurrentStep instance created.
2024-12-09 16:05:42,158 - stpipe.Detector1Pipeline.reset - INFO - ResetStep instance created.
2024-12-09 16:05:42,159 - stpipe.Detector1Pipeline.persistence - INFO - PersistenceStep instance created.
2024-12-09 16:05:42,160 - stpipe.Detector1Pipeline.charge_migration - INFO - ChargeMigrationStep instance created.
2024-12-09 16:05:42,162 - stpipe.Detector1Pipeline.jump - INFO - JumpStep instance created.
2024-12-09 16:05:42,163 - stpipe.Detector1Pipeline.ramp_fit - INFO - RampFitStep instance created.
2024-12-09 16:05:42,164 - stpipe.Detector1Pipeline.gain_scale - INFO - GainScaleStep instance created.
2024-12-09 16:05:42,318 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline running with args ('./nrc_im_demo_data/Obs001/uncal/jw02739001002_02105_00002_nrcalong_uncal.fits',).
2024-12-09 16:05:42,338 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage1
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_calibrated_ramp: False
  steps:
    group_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dq_init:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    emicorr:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      save_intermediate_results: False
      user_supplied_reffile: None
      nints_to_phase: None
      nbins: None
      scale_reference: True
      onthefly_corr_freq: None
      use_n_cycles: 3
    saturation:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      n_pix_grow_sat: 1
    ipc:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
    superbias:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    refpix:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      odd_even_columns: True
      use_side_ref_pixels: True
      side_smoothing_length: 11
      side_gain: 1.0
      odd_even_rows: True
      ovr_corr_mitigation_ftr: 3.0
      preserve_irs2_refpix: False
      irs2_mean_subtraction: False
    rscd:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      type: baseline
    firstframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    lastframe:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    linearity:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    dark_current:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      dark_output: None
      average_dark_current: None
    reset:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
    persistence:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      input_trapsfilled: ''
      flag_pers_cutoff: 40.0
      save_persistence: False
      save_trapsfilled: True
    charge_migration:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: True
      suffix: None
      search_output_file: True
      input_dir: ''
      signal_threshold: 25000.0
    jump:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      rejection_threshold: 4.0
      three_group_rejection_threshold: 6.0
      four_group_rejection_threshold: 5.0
      maximum_cores: half
      flag_4_neighbors: True
      max_jump_to_flag_neighbors: 1000
      min_jump_to_flag_neighbors: 30
      after_jump_flag_dn1: 0
      after_jump_flag_time1: 0
      after_jump_flag_dn2: 0
      after_jump_flag_time2: 0
      expand_large_events: True
      min_sat_area: 1
      min_jump_area: 5
      expand_factor: 2
      use_ellipses: False
      sat_required_snowball: True
      min_sat_radius_extend: 2.5
      sat_expand: 2
      edge_size: 25
      mask_snowball_core_next_int: True
      snowball_time_masked_next_int: 4000
      find_showers: False
      extend_snr_threshold: 0.0
      extend_min_area: 0
      extend_inner_radius: 0
      extend_outer_radius: 0.0
      extend_ellipse_expand_ratio: 0.0
      time_masked_after_shower: 0
      min_diffs_single_pass: 10
      max_extended_radius: 200
      minimum_groups: 3
      minimum_sigclip_groups: 100
      only_use_ints: True
    ramp_fit:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      algorithm: OLS_C
      int_name: ''
      save_opt: False
      opt_name: ''
      suppress_one_group: True
      maximum_cores: '1'
    gain_scale:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
2024-12-09 16:05:42,413 - stpipe.Detector1Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrcalong_uncal.fits' reftypes = ['dark', 'gain', 'linearity', 'mask', 'readnoise', 'refpix', 'reset', 'rscd', 'saturation', 'superbias']
2024-12-09 16:05:42,417 - stpipe.Detector1Pipeline - INFO - Prefetch for DARK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits'.
2024-12-09 16:05:42,417 - stpipe.Detector1Pipeline - INFO - Prefetch for GAIN reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits'.
2024-12-09 16:05:42,418 - stpipe.Detector1Pipeline - INFO - Prefetch for LINEARITY reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits'.
2024-12-09 16:05:42,418 - stpipe.Detector1Pipeline - INFO - Prefetch for MASK reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits'.
2024-12-09 16:05:42,418 - stpipe.Detector1Pipeline - INFO - Prefetch for READNOISE reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits'.
2024-12-09 16:05:42,419 - stpipe.Detector1Pipeline - INFO - Prefetch for REFPIX reference file is 'N/A'.
2024-12-09 16:05:42,419 - stpipe.Detector1Pipeline - INFO - Prefetch for RESET reference file is 'N/A'.
2024-12-09 16:05:42,419 - stpipe.Detector1Pipeline - INFO - Prefetch for RSCD reference file is 'N/A'.
2024-12-09 16:05:42,420 - stpipe.Detector1Pipeline - INFO - Prefetch for SATURATION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits'.
2024-12-09 16:05:42,420 - stpipe.Detector1Pipeline - INFO - Prefetch for SUPERBIAS reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits'.
2024-12-09 16:05:42,421 - stpipe.Detector1Pipeline - INFO - Starting calwebb_detector1 ...
2024-12-09 16:05:42,698 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:42,731 - stpipe.Detector1Pipeline.group_scale - INFO - NFRAMES and FRMDIVSR are equal; correction not needed
2024-12-09 16:05:42,732 - stpipe.Detector1Pipeline.group_scale - INFO - Step will be skipped
2024-12-09 16:05:42,733 - stpipe.Detector1Pipeline.group_scale - INFO - Step group_scale done
2024-12-09 16:05:42,871 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:42,889 - stpipe.Detector1Pipeline.dq_init - INFO - Using MASK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_mask_0076.fits
2024-12-09 16:05:43,052 - stpipe.Detector1Pipeline.dq_init - INFO - Step dq_init done
2024-12-09 16:05:43,197 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:43,216 - stpipe.Detector1Pipeline.saturation - INFO - Using SATURATION reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_saturation_0097.fits
2024-12-09 16:05:43,259 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword RESERVED_4 does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:43,302 - stpipe.Detector1Pipeline.saturation - WARNING - Keyword UNRELIABLE_RESET does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:44,164 - stpipe.Detector1Pipeline.saturation - INFO - Detected 26763 saturated pixels
2024-12-09 16:05:44,178 - stpipe.Detector1Pipeline.saturation - INFO - Detected 0 A/D floor pixels
2024-12-09 16:05:44,184 - stpipe.Detector1Pipeline.saturation - INFO - Step saturation done
2024-12-09 16:05:44,323 - stpipe.Detector1Pipeline.ipc - INFO - Step ipc running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:44,324 - stpipe.Detector1Pipeline.ipc - INFO - Step skipped.
2024-12-09 16:05:44,455 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:44,474 - stpipe.Detector1Pipeline.superbias - INFO - Using SUPERBIAS reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_superbias_0220.fits
2024-12-09 16:05:44,651 - stpipe.Detector1Pipeline.superbias - INFO - Step superbias done
2024-12-09 16:05:44,796 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:44,852 - stpipe.Detector1Pipeline.refpix - INFO - NIR full frame data
2024-12-09 16:05:44,853 - stpipe.Detector1Pipeline.refpix - INFO - The following parameters are valid for this mode:
2024-12-09 16:05:44,853 - stpipe.Detector1Pipeline.refpix - INFO - use_side_ref_pixels = True
2024-12-09 16:05:44,853 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_columns = True
2024-12-09 16:05:44,854 - stpipe.Detector1Pipeline.refpix - INFO - side_smoothing_length = 11
2024-12-09 16:05:44,854 - stpipe.Detector1Pipeline.refpix - INFO - side_gain = 1.0
2024-12-09 16:05:44,855 - stpipe.Detector1Pipeline.refpix - INFO - The following parameter is not applicable and is ignored:
2024-12-09 16:05:44,855 - stpipe.Detector1Pipeline.refpix - INFO - odd_even_rows = False
2024-12-09 16:05:46,407 - stpipe.Detector1Pipeline.refpix - INFO - Step refpix done
2024-12-09 16:05:46,583 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:46,604 - stpipe.Detector1Pipeline.linearity - INFO - Using Linearity reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_linearity_0052.fits
2024-12-09 16:05:46,676 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_DO_NOT_USE does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,677 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_LIN_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,677 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_BAD_LIN_FIT does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,678 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_NO_WELL_SAMP does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,679 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MODEL_FIT_FAIL does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,679 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_WELL_NOT_DEFINED does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:46,680 - stpipe.Detector1Pipeline.linearity - WARNING - Keyword UAZ_MASTER_MASK does not correspond to an existing DQ mnemonic, so will be ignored
2024-12-09 16:05:47,197 - stpipe.Detector1Pipeline.linearity - INFO - Step linearity done
2024-12-09 16:05:47,354 - stpipe.Detector1Pipeline.persistence - INFO - Step persistence running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:47,355 - stpipe.Detector1Pipeline.persistence - INFO - Step skipped.
2024-12-09 16:05:47,497 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:47,516 - stpipe.Detector1Pipeline.dark_current - INFO - Using DARK reference file /home/runner/crds/references/jwst/nircam/jwst_nircam_dark_0424.fits
2024-12-09 16:05:48,921 - stpipe.Detector1Pipeline.dark_current - INFO - Science data nints=1, ngroups=8, nframes=1, groupgap=1
2024-12-09 16:05:48,921 - stpipe.Detector1Pipeline.dark_current - INFO - Dark data nints=1, ngroups=130, nframes=1, groupgap=0
2024-12-09 16:05:49,382 - stpipe.Detector1Pipeline.dark_current - INFO - Step dark_current done
2024-12-09 16:05:49,537 - stpipe.Detector1Pipeline.charge_migration - INFO - Step charge_migration running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:49,538 - stpipe.Detector1Pipeline.charge_migration - INFO - Step skipped.
2024-12-09 16:05:49,678 - stpipe.Detector1Pipeline.jump - INFO - Step jump running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:05:49,688 - stpipe.Detector1Pipeline.jump - INFO - CR rejection threshold = 4 sigma
2024-12-09 16:05:49,689 - stpipe.Detector1Pipeline.jump - INFO - Maximum cores to use = half
2024-12-09 16:05:49,699 - stpipe.Detector1Pipeline.jump - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 16:05:49,721 - stpipe.Detector1Pipeline.jump - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 16:05:49,911 - stpipe.Detector1Pipeline.jump - INFO - Executing two-point difference method
2024-12-09 16:05:49,936 - stpipe.Detector1Pipeline.jump - INFO - Creating 8 processes for jump detection 
2024-12-09 16:05:53,183 - stpipe.Detector1Pipeline.jump - INFO - Flagging Snowballs
2024-12-09 16:06:03,392 - stpipe.Detector1Pipeline.jump - INFO - Total snowballs = 201
2024-12-09 16:06:03,392 - stpipe.Detector1Pipeline.jump - INFO - Total elapsed time = 13.4805 sec
2024-12-09 16:06:03,444 - stpipe.Detector1Pipeline.jump - INFO - The execution time in seconds: 13.755576
2024-12-09 16:06:03,446 - stpipe.Detector1Pipeline.jump - INFO - Step jump done
2024-12-09 16:06:03,603 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit running with args (<RampModel(1, 8, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:06:03,632 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using READNOISE reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_readnoise_0266.fits
2024-12-09 16:06:03,632 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using GAIN reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_gain_0097.fits
2024-12-09 16:06:03,660 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using algorithm = OLS_C
2024-12-09 16:06:03,661 - stpipe.Detector1Pipeline.ramp_fit - INFO - Using weighting = optimal
2024-12-09 16:06:03,916 - stpipe.Detector1Pipeline.ramp_fit - INFO - Number of multiprocessing slices: 1
2024-12-09 16:06:06,522 - stpipe.Detector1Pipeline.ramp_fit - INFO - Ramp Fitting C Time: 2.6011698246002197
2024-12-09 16:06:06,733 - stpipe.Detector1Pipeline.ramp_fit - INFO - Step ramp_fit done
2024-12-09 16:06:06,900 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<ImageModel(2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:06:06,929 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:06:06,929 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:06:06,931 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:06:07,090 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale running with args (<CubeModel(1, 2048, 2048) from jw02739001002_02105_00002_nrcalong_uncal.fits>,).
2024-12-09 16:06:07,119 - stpipe.Detector1Pipeline.gain_scale - INFO - GAINFACT not found in gain reference file
2024-12-09 16:06:07,120 - stpipe.Detector1Pipeline.gain_scale - INFO - Step will be skipped
2024-12-09 16:06:07,121 - stpipe.Detector1Pipeline.gain_scale - INFO - Step gain_scale done
2024-12-09 16:06:07,278 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rateints.fits
2024-12-09 16:06:07,279 - stpipe.Detector1Pipeline - INFO - ... ending calwebb_detector1
2024-12-09 16:06:07,280 - stpipe.Detector1Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:06:07,429 - stpipe.Detector1Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits
2024-12-09 16:06:07,430 - stpipe.Detector1Pipeline - INFO - Step Detector1Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Detector1: {time1 - time_det1:0.0f} seconds")
Runtime so far: 531 seconds
Runtime for Detector1: 517 seconds

Exploring the data#

Identify *_rate.fits files and verify which pipeline steps were run and which calibration reference files were applied.

The header contains information about which calibration steps were completed and skipped and which reference files were used to process the data.

if dodet1:
    # find rate files
    rate_files = sorted(glob.glob(os.path.join(det1_dir, '*_rate.fits')))

    # Read in file as datamodel
    rate_f = datamodels.open(rate_files[0])

    # Check which steps were run
    rate_f.meta.cal_step.instance
{'charge_migration': 'SKIPPED',
 'dark_sub': 'COMPLETE',
 'dq_init': 'COMPLETE',
 'gain_scale': 'SKIPPED',
 'group_scale': 'SKIPPED',
 'ipc': 'SKIPPED',
 'jump': 'COMPLETE',
 'linearity': 'COMPLETE',
 'persistence': 'SKIPPED',
 'ramp_fit': 'COMPLETE',
 'refpix': 'COMPLETE',
 'saturation': 'COMPLETE',
 'superbias': 'COMPLETE'}

For this particular rate file, show which reference files were used to calibrate the dataset. Note that these files will be different for each NIRCam detector.

if dodet1:
    rate_f.meta.ref_file.instance
{'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
 'dark': {'name': 'crds://jwst_nircam_dark_0342.fits'},
 'gain': {'name': 'crds://jwst_nircam_gain_0093.fits'},
 'linearity': {'name': 'crds://jwst_nircam_linearity_0051.fits'},
 'mask': {'name': 'crds://jwst_nircam_mask_0072.fits'},
 'readnoise': {'name': 'crds://jwst_nircam_readnoise_0224.fits'},
 'saturation': {'name': 'crds://jwst_nircam_saturation_0104.fits'},
 'superbias': {'name': 'crds://jwst_nircam_superbias_0186.fits'}}

6. Image2 Pipeline#

In the Image2 stage of the pipeline, calibrated unrectified data products are created (*_cal.fits or *_calints.fits files, depending on whether the input files are *_rate.fits or *_rateints.fits).

In this pipeline processing stage, the world coordinate system (WCS) is assigned, the data are flat fielded, and a photometric calibration is applied to convert from units of countrate (ADU/s) to surface brightness (MJy/sr).

By default, the background subtraction step and the resampling step are turned off for NIRCam. The background subtraction is turned off since there is no background template for the imaging mode and the local background is subtracted as part of the photometry perfoemd in the source catalog step in the Image3 pipeline.

The resampling step occurs during the Image3 stage by default.

While the resampling step can be run on individual images in the Image2 stage, e.g., to prepare for generating a source catalog for each image, the default behavior is to run the step only in the Image3 stage, where multiple images are combined into a final mosaic after the outlier detection step flags bad pixels.

To turn on the resampling step in the Image2 stage, uncomment the line in the dicitionary below which sets image2dict['resample']['skip'] = False

time_image2 = time.perf_counter()
# Set up a dictionary to define how the Image2 pipeline should be configured.

# Boilerplate dictionary setup
image2dict = {}
image2dict['assign_wcs'], image2dict['flat_field'] = {}, {}
image2dict['photom'], image2dict['resample'] = {}, {}

# Overrides for whether or not certain steps should be skipped (example)
#image2dict['resample']['skip'] = False

# Overrides for various reference files
# Files should be in the base local directory or provide full path
#image2dict['assign_wcs']['override_distortion'] = 'myfile.asdf' # Spatial distortion (ASDF file)
#image2dict['assign_wcs']['override_filteroffset'] = 'myfile.asdf' # Imager filter offsets (ASDF file)
#image2dict['assign_wcs']['override_specwcs'] = 'myfile.asdf' # Spectral distortion (ASDF file)
#image2dict['assign_wcs']['override_wavelengthrange'] = 'myfile.asdf' # Wavelength channel mapping (ASDF file)
#image2dict['flat_field']['override_flat'] = 'myfile.fits' # Pixel flatfield
#image2dict['photom']['override_photom'] = 'myfile.fits' # Photometric calibration array

Find and sort all of the input files, ensuring use of absolute paths

sstring = os.path.join(det1_dir, 'jw*rate.fits')  # Use files from the detector1 output folder
rate_files = sorted(glob.glob(sstring))
rate_files = [os.path.abspath(fname) for fname in rate_files]

print(f"Found  {len(rate_files)} science files")
Found  6 science files
# List rate files
rate_files
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits']

Run the Image2 pipeline on all of the rate files, regardless of filter. Note that if you have exposures with multiple integrations and you wish to keep the integrations separate, you should call the pipeline on the *rateints.fits files, rather than the *rate.fits files.

# Run Image2 stage of pipeline, specifying:
# output directory to save *_cal.fits files
# save_results flag set to True so the rate files are saved

if doimage2:
    for rate in rate_files:
        cal_result = Image2Pipeline.call(rate, output_dir=image2_dir, steps=image2dict, save_results=True)
else:
    print("Skipping Image2 processing.")
2024-12-09 16:06:07,577 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf    1.1 K bytes  (1 / 1 files) (0 / 1.1 K bytes)
2024-12-09 16:06:07,651 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:06:07,666 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:06:07,667 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:06:07,669 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:06:07,670 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:06:07,671 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:06:07,672 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:06:07,853 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits',).
2024-12-09 16:06:07,861 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:06:07,913 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:06:07,917 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits   16.8 M bytes  (1 / 5 files) (0 / 67.2 M bytes)
2024-12-09 16:06:08,289 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf   14.3 K bytes  (2 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:08,358 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf   11.4 K bytes  (3 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:08,432 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits   50.4 M bytes  (4 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:09,606 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits   23.0 K bytes  (5 / 5 files) (67.2 M / 67.2 M bytes)
2024-12-09 16:06:09,687 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits'.
2024-12-09 16:06:09,688 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:06:09,688 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:06:09,689 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:06:09,689 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:06:09,689 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf'.
2024-12-09 16:06:09,690 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:06:09,691 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 16:06:09,691 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits'.
2024-12-09 16:06:09,692 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:06:09,692 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:06:09,692 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:06:09,693 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:06:09,693 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:06:09,693 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:06:09,694 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:06:09,694 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits'.
2024-12-09 16:06:09,694 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:06:09,695 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:06:09,695 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:06:09,695 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:06:09,696 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:06:09,697 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:06:09,697 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2
2024-12-09 16:06:09,697 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2_rate.fits ...
2024-12-09 16:06:09,893 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:10,014 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:06:10,014 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:10,014 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:10,150 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.711062770 -13.874490965 274.729108941 -13.875063132 274.729787198 -13.857677823 274.711707966 -13.856914583
2024-12-09 16:06:10,151 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.711062770 -13.874490965 274.729108941 -13.875063132 274.729787198 -13.857677823 274.711707966 -13.856914583
2024-12-09 16:06:10,152 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:06:10,154 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:06:10,154 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:06:10,155 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:10,155 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:10,156 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:06:10,157 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:06:10,157 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:06:10,157 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:10,198 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:06:10,354 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:10,435 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits
2024-12-09 16:06:10,436 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:06:10,437 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:06:10,437 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:06:10,554 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:06:10,719 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:10,744 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits
2024-12-09 16:06:10,744 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits
2024-12-09 16:06:10,771 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:06:10,772 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCA2
2024-12-09 16:06:10,772 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:06:10,773 - stpipe.Image2Pipeline.photom - INFO -  filter: F200W
2024-12-09 16:06:10,773 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:06:10,808 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:06:10,809 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:06:10,810 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 2.1
2024-12-09 16:06:10,829 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:06:10,985 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:11,003 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:06:11,004 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:06:11,004 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:06:11,005 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:06:11,005 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:06:11,032 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 16:06:11,086 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:06:12,801 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:13,624 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:06:15,307 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:17,900 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:20,442 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:21,364 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.711013782 -13.874493489 274.729096668 -13.875161576 274.729784767 -13.857589202 274.711703249 -13.856921167
2024-12-09 16:06:21,588 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_i2d.fits
2024-12-09 16:06:21,589 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:06:21,590 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca2
2024-12-09 16:06:21,591 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:06:21,591 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:06:21,849 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_cal.fits
2024-12-09 16:06:21,850 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 16:06:21,903 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:06:21,916 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:06:21,917 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:06:21,918 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:06:21,919 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:06:21,920 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:06:21,921 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:06:22,086 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits',).
2024-12-09 16:06:22,094 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:06:22,150 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca4_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:06:22,155 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits   16.8 M bytes  (1 / 4 files) (0 / 67.2 M bytes)
2024-12-09 16:06:22,529 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf   14.3 K bytes  (2 / 4 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:22,577 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits   50.4 M bytes  (3 / 4 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:23,894 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits   23.0 K bytes  (4 / 4 files) (67.2 M / 67.2 M bytes)
2024-12-09 16:06:23,979 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits'.
2024-12-09 16:06:23,979 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:06:23,980 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:06:23,980 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:06:23,981 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:06:23,981 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf'.
2024-12-09 16:06:23,982 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:06:23,982 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 16:06:23,982 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits'.
2024-12-09 16:06:23,983 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:06:23,983 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:06:23,983 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:06:23,984 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:06:23,984 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:06:23,984 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:06:23,985 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:06:23,985 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits'.
2024-12-09 16:06:23,985 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:06:23,985 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:06:23,986 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:06:23,986 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:06:23,986 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:06:23,987 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:06:23,988 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4
2024-12-09 16:06:23,988 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4_rate.fits ...
2024-12-09 16:06:24,216 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 16:06:24,336 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:06:24,337 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:24,337 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:24,468 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.711757205 -13.855753713 274.729845583 -13.856419657 274.730660309 -13.838961453 274.712484599 -13.838101450
2024-12-09 16:06:24,468 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.711757205 -13.855753713 274.729845583 -13.856419657 274.730660309 -13.838961453 274.712484599 -13.838101450
2024-12-09 16:06:24,469 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:06:24,471 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:06:24,471 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:06:24,472 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:24,472 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:24,473 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:06:24,473 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:06:24,473 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:06:24,474 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:24,495 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 16:06:24,496 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:24,496 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:24,519 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:06:24,676 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 16:06:24,757 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits
2024-12-09 16:06:24,758 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:06:24,758 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:06:24,759 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:06:24,892 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:06:25,049 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 16:06:25,073 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits
2024-12-09 16:06:25,074 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits
2024-12-09 16:06:25,101 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:06:25,101 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCA4
2024-12-09 16:06:25,102 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:06:25,102 - stpipe.Image2Pipeline.photom - INFO -  filter: F200W
2024-12-09 16:06:25,103 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:06:25,134 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:06:25,135 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:06:25,136 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 1.829
2024-12-09 16:06:25,153 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:06:25,313 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_image2pipeline.fits>,).
2024-12-09 16:06:25,332 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:06:25,332 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:06:25,333 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:06:25,333 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:06:25,334 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:06:25,361 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030907888443991595 arcsec.
2024-12-09 16:06:25,416 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:06:27,158 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:06:27,981 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:06:29,710 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:06:32,364 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:06:34,888 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:06:35,811 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.711689861 -13.855755202 274.729871311 -13.856521792 274.730660128 -13.838869390 274.712480056 -13.838102858
2024-12-09 16:06:36,026 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_i2d.fits
2024-12-09 16:06:36,026 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:06:36,027 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrca4
2024-12-09 16:06:36,028 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:06:36,028 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:06:36,287 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_cal.fits
2024-12-09 16:06:36,287 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 16:06:36,341 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:06:36,354 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:06:36,355 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:06:36,356 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:06:36,357 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:06:36,358 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:06:36,360 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:06:36,521 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits',).
2024-12-09 16:06:36,529 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:06:36,584 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:06:36,589 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits   16.8 M bytes  (1 / 5 files) (0 / 67.2 M bytes)
2024-12-09 16:06:36,985 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf   12.7 K bytes  (2 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:37,042 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf   11.4 K bytes  (3 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:37,094 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits   50.4 M bytes  (4 / 5 files) (16.8 M / 67.2 M bytes)
2024-12-09 16:06:38,622 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits   23.0 K bytes  (5 / 5 files) (67.2 M / 67.2 M bytes)
2024-12-09 16:06:38,678 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits'.
2024-12-09 16:06:38,679 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:06:38,680 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:06:38,680 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:06:38,680 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:06:38,681 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf'.
2024-12-09 16:06:38,681 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:06:38,682 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2024-12-09 16:06:38,682 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits'.
2024-12-09 16:06:38,683 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:06:38,683 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:06:38,683 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:06:38,684 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:06:38,684 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:06:38,684 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:06:38,684 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:06:38,685 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits'.
2024-12-09 16:06:38,685 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:06:38,685 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:06:38,686 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:06:38,686 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:06:38,687 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:06:38,687 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:06:38,688 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong
2024-12-09 16:06:38,688 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong_rate.fits ...
2024-12-09 16:06:38,907 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 16:06:39,025 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:06:39,026 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:39,026 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:39,161 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.692000981 -13.873604403 274.728935032 -13.874635765 274.730488193 -13.839266605 274.693322826 -13.837371261
2024-12-09 16:06:39,161 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.692000981 -13.873604403 274.728935032 -13.874635765 274.730488193 -13.839266605 274.693322826 -13.837371261
2024-12-09 16:06:39,162 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:06:39,164 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:06:39,165 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:06:39,165 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:39,166 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:39,166 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:06:39,166 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:06:39,167 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:06:39,167 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:39,216 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:06:39,393 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 16:06:39,477 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits
2024-12-09 16:06:39,478 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:06:39,478 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:06:39,479 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:06:39,610 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:06:39,789 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 16:06:39,815 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits
2024-12-09 16:06:39,815 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits
2024-12-09 16:06:39,843 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:06:39,844 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCALONG
2024-12-09 16:06:39,844 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:06:39,845 - stpipe.Image2Pipeline.photom - INFO -  filter: F444W
2024-12-09 16:06:39,845 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:06:39,877 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:06:39,877 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:06:39,879 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 0.402
2024-12-09 16:06:39,897 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:06:40,062 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_image2pipeline.fits>,).
2024-12-09 16:06:40,080 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:06:40,080 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:06:40,081 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:06:40,081 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:06:40,081 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:06:40,107 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 16:06:40,163 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:06:41,865 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:06:42,696 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:06:44,310 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:06:46,844 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:06:49,377 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:06:50,306 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.691795163 -13.873605296 274.728986649 -13.875071623 274.730499024 -13.838842900 274.693313328 -13.837376803
2024-12-09 16:06:50,534 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_i2d.fits
2024-12-09 16:06:50,534 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:06:50,535 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00001_nrcalong
2024-12-09 16:06:50,536 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:06:50,536 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:06:50,800 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_cal.fits
2024-12-09 16:06:50,800 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 16:06:50,855 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:06:50,868 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:06:50,869 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:06:50,871 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:06:50,872 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:06:50,873 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:06:50,875 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:06:51,072 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits',).
2024-12-09 16:06:51,082 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:06:51,139 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca2_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:06:51,144 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits'.
2024-12-09 16:06:51,144 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:06:51,145 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:06:51,145 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:06:51,145 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:06:51,146 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0268.asdf'.
2024-12-09 16:06:51,146 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:06:51,146 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 16:06:51,147 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits'.
2024-12-09 16:06:51,148 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:06:51,148 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:06:51,148 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:06:51,149 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:06:51,149 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:06:51,149 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:06:51,150 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:06:51,150 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits'.
2024-12-09 16:06:51,151 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:06:51,151 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:06:51,151 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:06:51,152 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:06:51,152 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:06:51,153 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:06:51,153 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2
2024-12-09 16:06:51,154 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2_rate.fits ...
2024-12-09 16:06:51,388 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:51,506 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:06:51,507 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:51,507 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:51,661 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.714148074 -13.874666536 274.732194265 -13.875238512 274.732872327 -13.857853196 274.714793074 -13.857090147
2024-12-09 16:06:51,661 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.714148074 -13.874666536 274.732194265 -13.875238512 274.732872327 -13.857853196 274.714793074 -13.857090147
2024-12-09 16:06:51,662 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:06:51,664 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:06:51,664 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:06:51,664 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:06:51,665 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:51,665 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:06:51,665 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:06:51,666 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:06:51,666 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:06:51,706 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:06:51,871 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:51,947 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0654.fits
2024-12-09 16:06:51,948 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:06:51,948 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:06:51,949 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:06:52,067 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:06:52,230 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:52,254 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0152.fits
2024-12-09 16:06:52,255 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0263.fits
2024-12-09 16:06:52,282 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:06:52,282 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCA2
2024-12-09 16:06:52,283 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:06:52,283 - stpipe.Image2Pipeline.photom - INFO -  filter: F200W
2024-12-09 16:06:52,284 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:06:52,315 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:06:52,316 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:06:52,317 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 2.1
2024-12-09 16:06:52,335 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:06:52,502 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_image2pipeline.fits>,).
2024-12-09 16:06:52,520 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:06:52,521 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:06:52,521 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:06:52,522 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:06:52,522 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:06:52,548 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030776005265217777 arcsec.
2024-12-09 16:06:52,603 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:06:54,299 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:55,121 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:06:56,782 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:06:59,325 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:07:01,873 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2055, 2057)
2024-12-09 16:07:02,790 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.714099086 -13.874669061 274.732181993 -13.875336956 274.732869896 -13.857764576 274.714788356 -13.857096731
2024-12-09 16:07:03,008 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_i2d.fits
2024-12-09 16:07:03,009 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:07:03,010 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca2
2024-12-09 16:07:03,010 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:07:03,011 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:07:03,266 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_cal.fits
2024-12-09 16:07:03,267 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 16:07:03,321 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:07:03,334 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:07:03,335 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:07:03,336 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:07:03,337 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:07:03,338 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:07:03,339 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:07:03,501 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits',).
2024-12-09 16:07:03,509 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:07:03,565 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrca4_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:07:03,570 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits'.
2024-12-09 16:07:03,570 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:07:03,571 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:07:03,571 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:07:03,571 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:07:03,572 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0313.asdf'.
2024-12-09 16:07:03,572 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:07:03,572 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0005.asdf'.
2024-12-09 16:07:03,573 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits'.
2024-12-09 16:07:03,573 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:07:03,573 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:07:03,574 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:07:03,574 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:07:03,574 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:07:03,575 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:07:03,575 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:07:03,575 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits'.
2024-12-09 16:07:03,576 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:07:03,576 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:07:03,576 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:07:03,576 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:07:03,577 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:07:03,577 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:07:03,578 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4
2024-12-09 16:07:03,580 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4_rate.fits ...
2024-12-09 16:07:03,792 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 16:07:03,913 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:07:03,913 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:07:03,914 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:04,045 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.714842299 -13.855929276 274.732930699 -13.856595030 274.733745230 -13.839136817 274.715569496 -13.838277006
2024-12-09 16:07:04,045 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.714842299 -13.855929276 274.732930699 -13.856595030 274.733745230 -13.839136817 274.715569496 -13.838277006
2024-12-09 16:07:04,046 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:07:04,048 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:07:04,048 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:07:04,048 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:07:04,049 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:04,049 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:07:04,049 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:07:04,050 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:07:04,050 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:04,072 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 16:07:04,072 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:07:04,073 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:04,096 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:07:04,262 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 16:07:04,339 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0653.fits
2024-12-09 16:07:04,340 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:07:04,340 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:07:04,341 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:07:04,469 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:07:04,659 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 16:07:04,686 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0159.fits
2024-12-09 16:07:04,686 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0290.fits
2024-12-09 16:07:04,715 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:07:04,716 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCA4
2024-12-09 16:07:04,716 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:07:04,716 - stpipe.Image2Pipeline.photom - INFO -  filter: F200W
2024-12-09 16:07:04,717 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:07:04,750 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:07:04,750 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:07:04,752 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 1.829
2024-12-09 16:07:04,771 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:07:04,964 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_image2pipeline.fits>,).
2024-12-09 16:07:04,983 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:07:04,983 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:07:04,984 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:07:04,984 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:07:04,985 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:07:05,011 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.030907888553463565 arcsec.
2024-12-09 16:07:05,068 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:07:06,789 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:07:07,611 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:07:09,253 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:07:11,815 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:07:14,406 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2058, 2058)
2024-12-09 16:07:15,349 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.714774955 -13.855930766 274.732956428 -13.856697164 274.733745047 -13.839044754 274.715564953 -13.838278414
2024-12-09 16:07:15,578 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_i2d.fits
2024-12-09 16:07:15,579 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:07:15,580 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrca4
2024-12-09 16:07:15,581 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:07:15,582 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:07:15,844 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_cal.fits
2024-12-09 16:07:15,845 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
2024-12-09 16:07:15,902 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:07:15,915 - stpipe.Image2Pipeline - INFO - Image2Pipeline instance created.
2024-12-09 16:07:15,916 - stpipe.Image2Pipeline.bkg_subtract - INFO - BackgroundStep instance created.
2024-12-09 16:07:15,917 - stpipe.Image2Pipeline.assign_wcs - INFO - AssignWcsStep instance created.
2024-12-09 16:07:15,918 - stpipe.Image2Pipeline.flat_field - INFO - FlatFieldStep instance created.
2024-12-09 16:07:15,919 - stpipe.Image2Pipeline.photom - INFO - PhotomStep instance created.
2024-12-09 16:07:15,920 - stpipe.Image2Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:07:16,111 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline running with args ('/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits',).
2024-12-09 16:07:16,119 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage2
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  save_bsub: False
  steps:
    bkg_subtract:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_combined_background: False
      sigma: 3.0
      maxiters: None
      wfss_mmag_extract: None
    assign_wcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
      slit_y_low: -0.55
      slit_y_high: 0.55
    flat_field:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_interpolated_flat: False
      user_supplied_flat: None
      inverse: False
    photom:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      inverse: False
      source_type: None
      mrs_time_correction: True
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
2024-12-09 16:07:16,175 - stpipe.Image2Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00002_nrcalong_rate.fits' reftypes = ['area', 'camera', 'collimator', 'dflat', 'disperser', 'distortion', 'fflat', 'filteroffset', 'flat', 'fore', 'fpa', 'ifufore', 'ifupost', 'ifuslicer', 'msa', 'ote', 'photom', 'regions', 'sflat', 'specwcs', 'wavelengthrange', 'wfssbkg']
2024-12-09 16:07:16,180 - stpipe.Image2Pipeline - INFO - Prefetch for AREA reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits'.
2024-12-09 16:07:16,181 - stpipe.Image2Pipeline - INFO - Prefetch for CAMERA reference file is 'N/A'.
2024-12-09 16:07:16,181 - stpipe.Image2Pipeline - INFO - Prefetch for COLLIMATOR reference file is 'N/A'.
2024-12-09 16:07:16,182 - stpipe.Image2Pipeline - INFO - Prefetch for DFLAT reference file is 'N/A'.
2024-12-09 16:07:16,182 - stpipe.Image2Pipeline - INFO - Prefetch for DISPERSER reference file is 'N/A'.
2024-12-09 16:07:16,183 - stpipe.Image2Pipeline - INFO - Prefetch for DISTORTION reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_distortion_0277.asdf'.
2024-12-09 16:07:16,183 - stpipe.Image2Pipeline - INFO - Prefetch for FFLAT reference file is 'N/A'.
2024-12-09 16:07:16,184 - stpipe.Image2Pipeline - INFO - Prefetch for FILTEROFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_filteroffset_0007.asdf'.
2024-12-09 16:07:16,184 - stpipe.Image2Pipeline - INFO - Prefetch for FLAT reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits'.
2024-12-09 16:07:16,184 - stpipe.Image2Pipeline - INFO - Prefetch for FORE reference file is 'N/A'.
2024-12-09 16:07:16,185 - stpipe.Image2Pipeline - INFO - Prefetch for FPA reference file is 'N/A'.
2024-12-09 16:07:16,185 - stpipe.Image2Pipeline - INFO - Prefetch for IFUFORE reference file is 'N/A'.
2024-12-09 16:07:16,186 - stpipe.Image2Pipeline - INFO - Prefetch for IFUPOST reference file is 'N/A'.
2024-12-09 16:07:16,186 - stpipe.Image2Pipeline - INFO - Prefetch for IFUSLICER reference file is 'N/A'.
2024-12-09 16:07:16,186 - stpipe.Image2Pipeline - INFO - Prefetch for MSA reference file is 'N/A'.
2024-12-09 16:07:16,187 - stpipe.Image2Pipeline - INFO - Prefetch for OTE reference file is 'N/A'.
2024-12-09 16:07:16,187 - stpipe.Image2Pipeline - INFO - Prefetch for PHOTOM reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits'.
2024-12-09 16:07:16,188 - stpipe.Image2Pipeline - INFO - Prefetch for REGIONS reference file is 'N/A'.
2024-12-09 16:07:16,188 - stpipe.Image2Pipeline - INFO - Prefetch for SFLAT reference file is 'N/A'.
2024-12-09 16:07:16,188 - stpipe.Image2Pipeline - INFO - Prefetch for SPECWCS reference file is 'N/A'.
2024-12-09 16:07:16,189 - stpipe.Image2Pipeline - INFO - Prefetch for WAVELENGTHRANGE reference file is 'N/A'.
2024-12-09 16:07:16,189 - stpipe.Image2Pipeline - INFO - Prefetch for WFSSBKG reference file is 'N/A'.
2024-12-09 16:07:16,190 - stpipe.Image2Pipeline - INFO - Starting calwebb_image2 ...
2024-12-09 16:07:16,190 - stpipe.Image2Pipeline - INFO - Processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong
2024-12-09 16:07:16,193 - stpipe.Image2Pipeline - INFO - Working on input /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong_rate.fits ...
2024-12-09 16:07:16,411 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 16:07:16,528 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:07:16,528 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:07:16,529 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:16,670 - stpipe.Image2Pipeline.assign_wcs - INFO - Update S_REGION to POLYGON ICRS  274.695086261 -13.873780175 274.732020351 -13.874811147 274.733573116 -13.839441971 274.696407700 -13.837547020
2024-12-09 16:07:16,671 - stpipe.Image2Pipeline.assign_wcs - INFO - assign_wcs updated S_REGION to POLYGON ICRS  274.695086261 -13.873780175 274.732020351 -13.874811147 274.733573116 -13.839441971 274.696407700 -13.837547020
2024-12-09 16:07:16,671 - stpipe.Image2Pipeline.assign_wcs - INFO - COMPLETED assign_wcs
2024-12-09 16:07:16,674 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:07:16,674 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use FITS instead.
2024-12-09 16:07:16,674 - stpipe.Image2Pipeline.assign_wcs - WARNING -   warnings.warn(
2024-12-09 16:07:16,675 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:16,675 - stpipe.Image2Pipeline.assign_wcs - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:07:16,675 - stpipe.Image2Pipeline.assign_wcs - WARNING -         Use to_string() instead.
2024-12-09 16:07:16,676 - stpipe.Image2Pipeline.assign_wcs - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:07:16,676 - stpipe.Image2Pipeline.assign_wcs - WARNING - 
2024-12-09 16:07:16,725 - stpipe.Image2Pipeline.assign_wcs - INFO - Step assign_wcs done
2024-12-09 16:07:16,907 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 16:07:16,986 - stpipe.Image2Pipeline.flat_field - INFO - Using FLAT reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_flat_0634.fits
2024-12-09 16:07:16,987 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type FFLAT
2024-12-09 16:07:16,987 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type SFLAT
2024-12-09 16:07:16,988 - stpipe.Image2Pipeline.flat_field - INFO - No reference found for type DFLAT
2024-12-09 16:07:17,111 - stpipe.Image2Pipeline.flat_field - INFO - Step flat_field done
2024-12-09 16:07:17,292 - stpipe.Image2Pipeline.photom - INFO - Step photom running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 16:07:17,317 - stpipe.Image2Pipeline.photom - INFO - Using photom reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_photom_0157.fits
2024-12-09 16:07:17,318 - stpipe.Image2Pipeline.photom - INFO - Using area reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_area_0261.fits
2024-12-09 16:07:17,345 - stpipe.Image2Pipeline.photom - INFO - Using instrument: NIRCAM
2024-12-09 16:07:17,345 - stpipe.Image2Pipeline.photom - INFO -  detector: NRCALONG
2024-12-09 16:07:17,345 - stpipe.Image2Pipeline.photom - INFO -  exp_type: NRC_IMAGE
2024-12-09 16:07:17,346 - stpipe.Image2Pipeline.photom - INFO -  filter: F444W
2024-12-09 16:07:17,346 - stpipe.Image2Pipeline.photom - INFO -  pupil: CLEAR
2024-12-09 16:07:17,378 - stpipe.Image2Pipeline.photom - INFO - Pixel area map copied to output.
2024-12-09 16:07:17,378 - stpipe.Image2Pipeline.photom - INFO - Values for PIXAR_SR and PIXAR_A2 obtained from AREA reference file.
2024-12-09 16:07:17,380 - stpipe.Image2Pipeline.photom - INFO - PHOTMJSR value: 0.402
2024-12-09 16:07:17,402 - stpipe.Image2Pipeline.photom - INFO - Step photom done
2024-12-09 16:07:17,581 - stpipe.Image2Pipeline.resample - INFO - Step resample running with args (<ImageModel(2048, 2048) from ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_image2pipeline.fits>,).
2024-12-09 16:07:17,599 - stpipe.Image2Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:07:17,599 - stpipe.Image2Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:07:17,600 - stpipe.Image2Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:07:17,600 - stpipe.Image2Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:07:17,601 - stpipe.Image2Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:07:17,627 - stpipe.Image2Pipeline.resample - INFO - Computed output pixel scale: 0.06290626050781634 arcsec.
2024-12-09 16:07:17,682 - stpipe.Image2Pipeline.resample - INFO - Resampling science data
2024-12-09 16:07:19,368 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:07:20,197 - stpipe.Image2Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:07:21,870 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:07:24,429 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:07:26,999 - stpipe.Image2Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2068, 2075)
2024-12-09 16:07:27,929 - stpipe.Image2Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.694880443 -13.873781071 274.732071973 -13.875247004 274.733583943 -13.839018266 274.696398203 -13.837552561
2024-12-09 16:07:28,150 - stpipe.Image2Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_i2d.fits
2024-12-09 16:07:28,151 - stpipe.Image2Pipeline.resample - INFO - Step resample done
2024-12-09 16:07:28,151 - stpipe.Image2Pipeline - INFO - Finished processing product /home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage1/jw02739001002_02105_00002_nrcalong
2024-12-09 16:07:28,152 - stpipe.Image2Pipeline - INFO - ... ending calwebb_image2
2024-12-09 16:07:28,153 - stpipe.Image2Pipeline - INFO - Results used CRDS context: jwst_1293.pmap
2024-12-09 16:07:28,413 - stpipe.Image2Pipeline - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_cal.fits
2024-12-09 16:07:28,414 - stpipe.Image2Pipeline - INFO - Step Image2Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Image2: {time1 - time_image2:0.0f} seconds")
Runtime so far: 612 seconds
Runtime for Image2: 81 seconds

Verify which pipeline steps were run#

if doimage2:
    # Identify *_cal.fits file products
    cal_files = sorted(glob.glob(os.path.join(image2_dir, '*_cal.fits')))

    # Select first file to gather information
    cal_f = datamodels.open(cal_files[0])

    # Check which steps were run:
    cal_f.meta.cal_step.instance
{'assign_wcs': 'COMPLETE',
 'charge_migration': 'SKIPPED',
 'dark_sub': 'COMPLETE',
 'dq_init': 'COMPLETE',
 'flat_field': 'COMPLETE',
 'gain_scale': 'SKIPPED',
 'group_scale': 'SKIPPED',
 'ipc': 'SKIPPED',
 'jump': 'COMPLETE',
 'linearity': 'COMPLETE',
 'persistence': 'SKIPPED',
 'photom': 'COMPLETE',
 'ramp_fit': 'COMPLETE',
 'refpix': 'COMPLETE',
 'saturation': 'COMPLETE',
 'superbias': 'COMPLETE'}

Check which reference files were used to calibrate the first file. Some of these will be detector-dependent.

if doimage2:
    cal_f.meta.ref_file.instance
{'area': {'name': 'crds://jwst_nircam_area_0263.fits'},
 'camera': {'name': 'N/A'},
 'collimator': {'name': 'N/A'},
 'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
 'dark': {'name': 'crds://jwst_nircam_dark_0342.fits'},
 'dflat': {'name': 'N/A'},
 'disperser': {'name': 'N/A'},
 'distortion': {'name': 'crds://jwst_nircam_distortion_0268.asdf'},
 'fflat': {'name': 'N/A'},
 'filteroffset': {'name': 'crds://jwst_nircam_filteroffset_0005.asdf'},
 'flat': {'name': 'crds://jwst_nircam_flat_0654.fits'},
 'fore': {'name': 'N/A'},
 'fpa': {'name': 'N/A'},
 'gain': {'name': 'crds://jwst_nircam_gain_0093.fits'},
 'ifufore': {'name': 'N/A'},
 'ifupost': {'name': 'N/A'},
 'ifuslicer': {'name': 'N/A'},
 'linearity': {'name': 'crds://jwst_nircam_linearity_0051.fits'},
 'mask': {'name': 'crds://jwst_nircam_mask_0072.fits'},
 'msa': {'name': 'N/A'},
 'ote': {'name': 'N/A'},
 'photom': {'name': 'crds://jwst_nircam_photom_0152.fits'},
 'readnoise': {'name': 'crds://jwst_nircam_readnoise_0224.fits'},
 'regions': {'name': 'N/A'},
 'saturation': {'name': 'crds://jwst_nircam_saturation_0104.fits'},
 'sflat': {'name': 'N/A'},
 'specwcs': {'name': 'N/A'},
 'superbias': {'name': 'crds://jwst_nircam_superbias_0186.fits'},
 'wavelengthrange': {'name': 'N/A'}}

7. Image3 Pipeline#

In the Image3 stage of the pipeline, the individual *_cal.fits files for each filter are combined to one single distortion corrected image. Unlike the previous stages, we must run the Image3 stage separately for the files from each filter as well as channel (i.e. shortwave vs longwave).

First, we need to create Associations, to inform the pipeline which files are linked together for each filter.

By default, the Image3 stage of the pipeline performs the following steps on NIRCam data:

  • tweakreg - creates source catalogs of pointlike sources for each input image. The source catalog for each input image is compared to each other to derive coordinate transforms to align the images relative to each other.

    • tweakreg has many input parameters that can be adjusted to improve the image alignment in cases where the default values do not perform well.

    • One tweakreg parameter that is not set by default but can be very useful is abs_refcat. When this parameter is set to GAIADR3, the tweakreg step performs an absolute astrometric correction of the data using the GAIA data release 3 catalog. In cases where multiple unsaturated GAIA stars are present in the input images, this can improve the absolute astrometric alignment. However, in sparse or very crowded fields, this can potentially result in poor performance, so users are encouraged to check astrometric accuracy and revisit this step if necessary.

    • As of pipeline version 1.14.0, the default source finding algorithm in the tweakreg step is IRAFStarFinder. Other options include DAOStarFinder, whose results are not as good in cases where the PSF is undersampled, such as in the blue filters of the NIRCam shortwave channel. Finally photutils segmentation SourceFinder, which does not assume sources are point-like.

  • skymatch - measures the background level from the sky to use as input into the subsequent outlier detection and resample steps.

  • outlier detection - flags any remaining cosmic rays, bad pixels, or other artifacts not already flagged during the Detector1 stage of the pipeline, using all input images to create a median image so that outliers in individual images can be identified.

  • resample - resamples each input image based on its WCS and distortion information and creates a single undistorted image.

  • source catalog - creates a catalog of detected sources along with photometric results and morphologies (i.e., point-like vs extended). These catalogs are generally useful for quick checks, but optimization is likely needed for specific science cases. Users may wish to experiment with changing the snr_threshold and deblend options. Modifications to the following parameters will not significantly improve data quality and it is advised to keep them at their default values: aperture_ee1, aperture_ee2, aperture_ee3, ci1_star_threshold, ci2_star_threshold.

time_image3 = time.perf_counter()
# Set up a dictionary to define how the Image3 pipeline should be configured

# Boilerplate dictionary setup
image3dict = {}
image3dict['assign_mtwcs'], image3dict['tweakreg'], image3dict['skymatch'] = {}, {}, {}
image3dict['outlier_detection'], image3dict['resample'], image3dict['source_catalog'] = {}, {}, {}

# Overrides for whether or not certain steps should be skipped (example)
#image3dict['outlier_detection']['skip'] = True

# Overrides for various reference files
# Files should be in the base local directory or provide full path
#image3dict['source_catalog']['override_apcorr'] = 'myfile.fits'  # Aperture correction parameters
#image3dict['source_catalog']['override_abvegaoffset'] = 'myfile.asdf'  # Data to convert from AB to Vega magnitudes (ASDF file)

# Turn on alignment to GAIA in the tweakreg step
# For data such as these demo data, where there are some heavily saturated stars in the field
# of view, alignment to GAIA sometimes does not work well due to tweakreg doing a poor job
# finding the centroids of the sources.
#image3dict['tweakreg']['abs_refcat'] = 'GAIADR3'

Find and sort all of the input files, ensuring use of absolute paths. Keep files for the two filters separated.

# Science Files need the cal.fits files
sw_sstring = os.path.join(image2_dir, 'jw*nrc??_cal.fits')     # shortwave files. Detectors a1-a4, b1-b4
lw_sstring = os.path.join(image2_dir, 'jw*nrc*long_cal.fits')  # longwave files. Detectors along, blong 

# Identify SW and LW cal files
sw_cal_files = sorted(glob.glob(sw_sstring))
lw_cal_files = sorted(glob.glob(lw_sstring))

# Expand the relative paths into absolute paths
sw_cal_files = [os.path.abspath(fname) for fname in sw_cal_files]
lw_cal_files = [os.path.abspath(fname) for fname in lw_cal_files]

print(f'Found {len(sw_cal_files)} shortwave science files to process')
print(f'Found {len(lw_cal_files)} longwave science files to process')
Found 4 shortwave science files to process
Found 2 longwave science files to process

Create Association File#

An association file lists the files to calibrate together in Stage3 of the pipeline. Note that association files are available for download from MAST, with filenames of *_asn.json. Here we show how to create an association file to point to the data products created in the steps above. This is useful in cases where you want to work with a set of data that is different than that in the association files from MAST.

Note that the output products will have a rootname that is specified by the product_name in the association file. For this tutorial, the rootnames of the output products will be image3_sw for filter F200W and image3_lw for filter F444W.

# List of data to use
print('List of SW cal files to use:')
sw_cal_files
print('\nList of LW cal files to use:')
lw_cal_files
List of SW cal files to use:
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca2_cal.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrca4_cal.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca2_cal.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrca4_cal.fits']
List of LW cal files to use:
['/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00001_nrcalong_cal.fits',
 '/home/runner/work/jwst-pipeline-notebooks/jwst-pipeline-notebooks/notebooks/NIRCAM/Imaging/nrc_im_demo_data/Obs001/stage2/jw02739001002_02105_00002_nrcalong_cal.fits']
# Create Level 3 Association for SW products
do_swimage3 = False
if doimage3:
    if len(sw_cal_files) > 0:
        # Only create an association file if there are SW data files to process
        do_swimage3 = True
        sw_product_name = 'image3_sw'
        sw_association = asn_from_list.asn_from_list(sw_cal_files,
                                                     rule=DMS_Level3_Base,
                                                     product_name=sw_product_name)
    
        sw_association.data['asn_type'] = 'image3'
        program = datamodels.open(sw_cal_files[0]).meta.observation.program_number
        sw_association.data['program'] = program
    
        # Format association as .json file
        sw_asn_filename, sw_serialized = sw_association.dump(format="json")

        # Write out association file
        sw_association_im3 = os.path.join(sci_dir, sw_asn_filename)
        with open(sw_association_im3, "w") as fd:
            fd.write(sw_serialized)
2024-12-09 16:07:28,697 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:07:28,697 - stpipe - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:07:28,698 - stpipe - WARNING - 
1533
# Create Level 3 Associations for LW products
do_lwimage3 = False
if doimage3:
    if len(lw_cal_files) > 0:
        # Only create an association file if there are SW data files to process
        do_lwimage3 = True
        lw_product_name = 'image3_lw'
        lw_association = asn_from_list.asn_from_list(lw_cal_files,
                                                     rule=DMS_Level3_Base,
                                                     product_name=lw_product_name)
    
        lw_association.data['asn_type'] = 'image3'
        program = datamodels.open(lw_cal_files[0]).meta.observation.program_number
        lw_association.data['program'] = program
    
        # Format association as .json file
        lw_asn_filename, lw_serialized = lw_association.dump(format="json")

        # Write out association file. Note that you can use your own filename in
        # place of lw_asn_filename and everything will still work.
        lw_association_im3 = os.path.join(sci_dir, lw_asn_filename)
        with open(lw_association_im3, "w") as fd:
            fd.write(lw_serialized)
2024-12-09 16:07:28,828 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:07:28,828 - stpipe - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:07:28,829 - stpipe - WARNING - 
989

Run Image3 stage of the pipeline#

For each set of grouped exposures in an association file, the Image3 stage of the pipeline will produce:

  • a *_crf.fits file produced by the outlier_detection step, where the DQ array marks the pixels flagged as outliers.

  • a final combined, rectified image with name *_i2d.fits,

  • a source catalog with name *_cat.ecsv,

  • a segmentation map file (*_segm.fits) which has integer values at the pixel locations where a source is detected where the pixel values match the source ID number in the catalog.

Run Image3 on the LW data#

# Run Stage3 on the LW data
if doimage3 and do_lwimage3:
    lw_i2d_result = Image3Pipeline.call(lw_association_im3, output_dir=image3_dir, steps=image3dict, save_results=True)
else:
    print('Skipping Image3 LW processing')
2024-12-09 16:07:29,024 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0045.asdf    1.6 K bytes  (1 / 1 files) (0 / 1.6 K bytes)
2024-12-09 16:07:29,087 - stpipe - INFO - PARS-TWEAKREGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0045.asdf
2024-12-09 16:07:29,099 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:07:29,108 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0001.asdf      992 bytes  (1 / 1 files) (0 / 992 bytes)
2024-12-09 16:07:29,157 - stpipe - INFO - PARS-SOURCECATALOGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0001.asdf
2024-12-09 16:07:29,173 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2024-12-09 16:07:29,174 - stpipe.Image3Pipeline.assign_mtwcs - INFO - AssignMTWcsStep instance created.
2024-12-09 16:07:29,177 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2024-12-09 16:07:29,178 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2024-12-09 16:07:29,179 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2024-12-09 16:07:29,180 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:07:29,182 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2024-12-09 16:07:29,380 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('./nrc_im_demo_data/Obs001/jw02739-a3001_image3_00009_asn.json',).
2024-12-09 16:07:29,394 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage3
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  steps:
    assign_mtwcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: True
      output_use_index: True
      save_results: False
      skip: False
      suffix: assign_mtwcs
      search_output_file: True
      input_dir: ''
    tweakreg:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: True
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_catalogs: False
      use_custom_catalogs: False
      catalog_format: ecsv
      catfile: ''
      starfinder: iraf
      snr_threshold: 10.0
      bkg_boxsize: 400
      kernel_fwhm: 2.302
      minsep_fwhm: 0.0
      sigma_radius: 1.5
      sharplo: 0.2
      sharphi: 1.0
      roundlo: -1.0
      roundhi: 1.0
      brightest: 200
      peakmax: None
      npixels: 10
      connectivity: '8'
      nlevels: 32
      contrast: 0.001
      multithresh_mode: exponential
      localbkg_width: 0
      apermask_method: correct
      kron_params: None
      enforce_user_order: False
      expand_refcat: False
      minobj: 15
      fitgeometry: shift
      nclip: 3
      sigma: 3.0
      searchrad: 2.0
      use2dhist: True
      separation: 2.0
      tolerance: 1.0
      xoffset: 0.0
      yoffset: 0.0
      abs_refcat: ''
      save_abs_catalog: False
      abs_minobj: 15
      abs_fitgeometry: rshift
      abs_nclip: 3
      abs_sigma: 3.0
      abs_searchrad: 6.0
      abs_use2dhist: True
      abs_separation: 1.0
      abs_tolerance: 0.7
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
    skymatch:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      skymethod: match
      match_down: True
      subtract: False
      stepsize: None
      skystat: mode
      dqbits: ~DO_NOT_USE+NON_SCIENCE
      lower: None
      upper: None
      nclip: 5
      lsigma: 4.0
      usigma: 4.0
      binwidth: 0.1
    outlier_detection:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: False
      input_dir: ''
      weight_type: ivm
      pixfrac: 1.0
      kernel: square
      fillval: INDEF
      nlow: 0
      nhigh: 0
      maskpt: 0.7
      snr: 5.0 4.0
      scale: 1.2 0.7
      backg: 0.0
      kernel_size: 7 7
      threshold_percent: 99.8
      rolling_window_width: 25
      ifu_second_check: False
      save_intermediate_results: False
      resample_data: True
      good_bits: ~DO_NOT_USE
      allowed_memory: None
      in_memory: False
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
    source_catalog:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: cat
      search_output_file: True
      input_dir: ''
      bkg_boxsize: 1000
      kernel_fwhm: 2.302
      snr_threshold: 3.0
      npixels: 25
      deblend: False
      aperture_ee1: 30
      aperture_ee2: 50
      aperture_ee3: 70
      ci1_star_threshold: 2.0
      ci2_star_threshold: 1.8
2024-12-09 16:07:29,406 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:07:29,407 - stpipe.Image3Pipeline - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:07:29,407 - stpipe.Image3Pipeline - WARNING - 
2024-12-09 16:07:29,548 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrcalong_cal.fits' reftypes = ['abvegaoffset', 'apcorr']
2024-12-09 16:07:29,552 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf    4.3 K bytes  (1 / 2 files) (0 / 24.5 K bytes)
2024-12-09 16:07:29,604 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits   20.2 K bytes  (2 / 2 files) (4.3 K / 24.5 K bytes)
2024-12-09 16:07:29,698 - stpipe.Image3Pipeline - INFO - Prefetch for ABVEGAOFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf'.
2024-12-09 16:07:29,699 - stpipe.Image3Pipeline - INFO - Prefetch for APCORR reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits'.
2024-12-09 16:07:29,700 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2024-12-09 16:07:29,714 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:07:29,715 - stpipe.Image3Pipeline - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:07:29,715 - stpipe.Image3Pipeline - WARNING - 
2024-12-09 16:07:30,160 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg running with args (<ModelContainer>,).
2024-12-09 16:07:31,714 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrcalong_cal.fits.
2024-12-09 16:07:33,286 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrcalong_cal.fits.
2024-12-09 16:07:33,309 - stpipe.Image3Pipeline.tweakreg - INFO - 
2024-12-09 16:07:33,309 - stpipe.Image3Pipeline.tweakreg - INFO - Number of image groups to be aligned: 2.
2024-12-09 16:07:33,310 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:07:33,310 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() started on 2024-12-09 16:07:33.310195
2024-12-09 16:07:33,310 - stpipe.Image3Pipeline.tweakreg - INFO -       Version 0.8.9
2024-12-09 16:07:33,311 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:07:33,806 - stpipe.Image3Pipeline.tweakreg - INFO - Selected image 'GROUP ID: jw02739001002_02105_1' as reference image
2024-12-09 16:07:33,812 - stpipe.Image3Pipeline.tweakreg - INFO - Aligning image catalog 'GROUP ID: jw02739001002_02105_2' to the reference catalog.
2024-12-09 16:07:33,854 - stpipe.Image3Pipeline.tweakreg - INFO - Matching sources from 'jw02739001002_02105_00002_nrcalong_cal' catalog with sources from the reference 'jw02739001002_02105_00001_nrcalong_cal' catalog.
2024-12-09 16:07:33,855 - stpipe.Image3Pipeline.tweakreg - INFO - Computing initial guess for X and Y shifts...
2024-12-09 16:07:33,856 - stpipe.Image3Pipeline.tweakreg - INFO - Found initial X and Y shifts of 0.013, 0.013 (arcsec) with significance of 121 and 121 matches.
2024-12-09 16:07:33,858 - stpipe.Image3Pipeline.tweakreg - INFO - Found 117 matches for 'GROUP ID: jw02739001002_02105_2'...
2024-12-09 16:07:33,858 - stpipe.Image3Pipeline.tweakreg - INFO - Performing 'shift' fit
2024-12-09 16:07:33,860 - stpipe.Image3Pipeline.tweakreg - INFO - Computed 'shift' fit for GROUP ID: jw02739001002_02105_2:
2024-12-09 16:07:33,861 - stpipe.Image3Pipeline.tweakreg - INFO - XSH: -0.00263329  YSH: 0.00178288
2024-12-09 16:07:33,861 - stpipe.Image3Pipeline.tweakreg - INFO - 
2024-12-09 16:07:33,862 - stpipe.Image3Pipeline.tweakreg - INFO - FIT RMSE: 0.0408995   FIT MAE: 0.0127711
2024-12-09 16:07:33,862 - stpipe.Image3Pipeline.tweakreg - INFO - Final solution based on 113 objects.
2024-12-09 16:07:33,895 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:07:33,896 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() ended on 2024-12-09 16:07:33.895905
2024-12-09 16:07:33,896 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() TOTAL RUN TIME: 0:00:00.585710
2024-12-09 16:07:33,897 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:07:33,958 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS  274.695085507 -13.873779680 274.732019598 -13.874810652 274.733572363 -13.839441476 274.696406947 -13.837546524
2024-12-09 16:07:33,961 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:07:33,961 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use FITS instead.
2024-12-09 16:07:33,961 - stpipe.Image3Pipeline.tweakreg - WARNING -   warnings.warn(
2024-12-09 16:07:33,962 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:07:33,962 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:07:33,962 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use to_string() instead.
2024-12-09 16:07:33,963 - stpipe.Image3Pipeline.tweakreg - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:07:33,963 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:07:34,025 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done
2024-12-09 16:07:34,209 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (<ModelContainer>,).
2024-12-09 16:07:34,258 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:07:34,259 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2024-12-09 16:07:34.258937
2024-12-09 16:07:34,259 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:07:34,260 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'match'
2024-12-09 16:07:34,260 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN
2024-12-09 16:07:34,261 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF
2024-12-09 16:07:34,261 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:07:34,261 - stpipe.Image3Pipeline.skymatch - INFO - ----  Computing differences in sky values in overlapping regions.
2024-12-09 16:07:35,266 - stpipe.Image3Pipeline.skymatch - INFO -    *  Image ID=jw02739001002_02105_00001_nrcalong_cal.fits. Sky background: 0
2024-12-09 16:07:35,267 - stpipe.Image3Pipeline.skymatch - INFO -    *  Image ID=jw02739001002_02105_00002_nrcalong_cal.fits. Sky background: 0.0721858
2024-12-09 16:07:35,267 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:07:35,268 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() ended on 2024-12-09 16:07:35.267825
2024-12-09 16:07:35,268 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() TOTAL RUN TIME: 0:00:01.008888
2024-12-09 16:07:35,268 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:07:35,273 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch done
2024-12-09 16:07:35,448 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection running with args (<ModelContainer>,).
2024-12-09 16:07:35,454 - stpipe.Image3Pipeline.outlier_detection - INFO - Performing outlier detection on 2 inputs
2024-12-09 16:07:35,455 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter kernel: square
2024-12-09 16:07:35,455 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:07:35,456 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter fillval: INDEF
2024-12-09 16:07:35,456 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter weight_type: ivm
2024-12-09 16:07:35,457 - stpipe.Image3Pipeline.outlier_detection - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:07:35,491 - stpipe.Image3Pipeline.outlier_detection - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 16:07:35,548 - stpipe.Image3Pipeline.outlier_detection - INFO - 1 exposures to drizzle together
2024-12-09 16:07:37,252 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:07:38,236 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_outlier_i2d.fits
2024-12-09 16:07:38,242 - stpipe.Image3Pipeline.outlier_detection - INFO - 1 exposures to drizzle together
2024-12-09 16:07:40,396 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:07:41,383 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_outlier_i2d.fits
2024-12-09 16:07:41,510 - stpipe.Image3Pipeline.outlier_detection - INFO - Computing median
2024-12-09 16:07:46,155 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_outlier_i2d.fits
2024-12-09 16:07:46,167 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_outlier_i2d.fits
2024-12-09 16:07:46,171 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting median
2024-12-09 16:07:47,935 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2240, 2078)
2024-12-09 16:07:48,271 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_blot.fits
2024-12-09 16:07:50,467 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2240, 2078)
2024-12-09 16:07:50,827 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_blot.fits
2024-12-09 16:07:50,828 - stpipe.Image3Pipeline.outlier_detection - INFO - Flagging outliers
2024-12-09 16:07:51,141 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 484 (0.01%)
2024-12-09 16:07:51,483 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 186 (0.00%)
2024-12-09 16:07:51,509 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_blot.fits
2024-12-09 16:07:51,535 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_blot.fits
2024-12-09 16:07:51,793 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrcalong_a3001_crf.fits
2024-12-09 16:07:52,076 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrcalong_a3001_crf.fits
2024-12-09 16:07:52,077 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection done
2024-12-09 16:07:52,264 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (<ModelContainer>,).
2024-12-09 16:07:52,271 - stpipe.Image3Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:07:52,271 - stpipe.Image3Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:07:52,271 - stpipe.Image3Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:07:52,272 - stpipe.Image3Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:07:52,272 - stpipe.Image3Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:07:52,308 - stpipe.Image3Pipeline.resample - INFO - Computed output pixel scale: 0.0629062603885582 arcsec.
2024-12-09 16:07:52,362 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for None
2024-12-09 16:07:52,910 - stpipe.Image3Pipeline.resample - INFO - Resampling science data
2024-12-09 16:07:54,682 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:07:57,721 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:07:58,555 - stpipe.Image3Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:08:00,224 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:02,778 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:05,373 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:08,376 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:11,489 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:14,508 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2240, 2078)
2024-12-09 16:08:15,479 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.691792687 -13.873658631 274.732077488 -13.875246915 274.733591808 -13.838965804 274.693313288 -13.837377769
2024-12-09 16:08:16,045 - stpipe.Image3Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_lw_i2d.fits
2024-12-09 16:08:16,045 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2024-12-09 16:08:16,274 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (<ImageModel(2240, 2078) from image3_lw_i2d.fits>,).
2024-12-09 16:08:16,292 - stpipe.Image3Pipeline.source_catalog - INFO - Using APCORR reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits
2024-12-09 16:08:16,300 - stpipe.Image3Pipeline.source_catalog - INFO - Using ABVEGAOFFSET reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf
2024-12-09 16:08:16,301 - stpipe.Image3Pipeline.source_catalog - INFO - Instrument: NIRCAM
2024-12-09 16:08:16,301 - stpipe.Image3Pipeline.source_catalog - INFO - Detector: NRCALONG
2024-12-09 16:08:16,302 - stpipe.Image3Pipeline.source_catalog - INFO - Filter: F444W
2024-12-09 16:08:16,302 - stpipe.Image3Pipeline.source_catalog - INFO - Pupil: CLEAR
2024-12-09 16:08:16,303 - stpipe.Image3Pipeline.source_catalog - INFO - Subarray: FULL
2024-12-09 16:08:16,326 - stpipe.Image3Pipeline.source_catalog - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:08:16,327 - stpipe.Image3Pipeline.source_catalog - WARNING -   warnings.warn(
2024-12-09 16:08:16,328 - stpipe.Image3Pipeline.source_catalog - WARNING - 
2024-12-09 16:08:16,347 - stpipe.Image3Pipeline.source_catalog - INFO - AB to Vega magnitude offset 3.23800
2024-12-09 16:08:19,667 - stpipe.Image3Pipeline.source_catalog - INFO - Detected 1489 sources
2024-12-09 16:08:21,036 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote source catalog: ./nrc_im_demo_data/Obs001/stage3/image3_lw_cat.ecsv
2024-12-09 16:08:21,186 - stpipe.Image3Pipeline.source_catalog - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_lw_segm.fits
2024-12-09 16:08:21,187 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote segmentation map: image3_lw_segm.fits
2024-12-09 16:08:21,190 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog done
2024-12-09 16:08:21,191 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline done

Some users wish to resample data from multiple filters onto the same WCS and pixel grid in order to create color images or help with subsequent analyses. In order to do that, we’ll save the gWCS from the *i2d.fits file created with the LW data above. The gWCS will be saved into an asdf file.

if doimage3 and do_lwimage3:
    # First we identify the dataset and read it using datamodels.
    lw_i2d_file = os.path.join(image3_dir, f'{lw_product_name}_i2d.fits')
    lw_data = datamodels.open(lw_i2d_file)
    
    # Pull out the resulting gWCS and save it in an asdf file
    tree = {"wcs": lw_data.meta.wcs}
    wcs_file = AsdfFile(tree)
    gwcs_filename = os.path.join(image3_dir + 'lw_gwcs.asdf')
    print(f'Saving gWCS into {gwcs_filename}')
    wcs_file.write_to(gwcs_filename)

    # Get the size of the mosaic image
    ysize, xsize = lw_data.data.shape
Saving gWCS into ./nrc_im_demo_data/Obs001/stage3lw_gwcs.asdf

Run Image3 on the SW data#

Prepare to call the Image3 pipeline on the SW data. If you wish to resample the SW data onto the same pixel grid as the LW data above, uncomment the lines below. This will tell the resample step to use the gWCS and the array size from the LW data when resampling the SW data.

# Uncoment this cell in order to resample the SW data onto the same pixel grid as the LW data
#if doimage3:
#    image3dict['resample']['output_wcs'] = gwcs_filename
#    image3dict['resample']['output_shape'] = (xsize, ysize)
if doimage3 and do_swimage3:
    sw_i2d_result = Image3Pipeline.call(sw_association_im3, output_dir=image3_dir, steps=image3dict, save_results=True)
else:
    print('Skipping Image3 SW processing')
2024-12-09 16:08:21,544 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:08:21,544 - stpipe - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:08:21,545 - stpipe - WARNING - 
2024-12-09 16:08:21,709 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0036.asdf    1.6 K bytes  (1 / 1 files) (0 / 1.6 K bytes)
2024-12-09 16:08:21,772 - stpipe - INFO - PARS-TWEAKREGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-tweakregstep_0036.asdf
2024-12-09 16:08:21,786 - stpipe - INFO - PARS-RESAMPLESTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-resamplestep_0001.asdf
2024-12-09 16:08:21,796 - CRDS - INFO -  Fetching  /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0007.asdf      992 bytes  (1 / 1 files) (0 / 992 bytes)
2024-12-09 16:08:21,846 - stpipe - INFO - PARS-SOURCECATALOGSTEP parameters found: /home/runner/crds/references/jwst/nircam/jwst_nircam_pars-sourcecatalogstep_0007.asdf
2024-12-09 16:08:21,863 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2024-12-09 16:08:21,864 - stpipe.Image3Pipeline.assign_mtwcs - INFO - AssignMTWcsStep instance created.
2024-12-09 16:08:21,867 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2024-12-09 16:08:21,868 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2024-12-09 16:08:21,870 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2024-12-09 16:08:21,871 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2024-12-09 16:08:21,872 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2024-12-09 16:08:22,099 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('./nrc_im_demo_data/Obs001/jw02739-a3001_image3_00008_asn.json',).
2024-12-09 16:08:22,113 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: ./nrc_im_demo_data/Obs001/stage3
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
  steps:
    assign_mtwcs:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: True
      output_use_index: True
      save_results: False
      skip: False
      suffix: assign_mtwcs
      search_output_file: True
      input_dir: ''
    tweakreg:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: True
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      save_catalogs: False
      use_custom_catalogs: False
      catalog_format: ecsv
      catfile: ''
      starfinder: iraf
      snr_threshold: 10.0
      bkg_boxsize: 400
      kernel_fwhm: 2.141
      minsep_fwhm: 0.0
      sigma_radius: 1.5
      sharplo: 0.2
      sharphi: 1.0
      roundlo: -1.0
      roundhi: 1.0
      brightest: 200
      peakmax: None
      npixels: 10
      connectivity: '8'
      nlevels: 32
      contrast: 0.001
      multithresh_mode: exponential
      localbkg_width: 0
      apermask_method: correct
      kron_params: None
      enforce_user_order: False
      expand_refcat: False
      minobj: 15
      fitgeometry: shift
      nclip: 3
      sigma: 3.0
      searchrad: 2.0
      use2dhist: True
      separation: 2.0
      tolerance: 1.0
      xoffset: 0.0
      yoffset: 0.0
      abs_refcat: ''
      save_abs_catalog: False
      abs_minobj: 15
      abs_fitgeometry: rshift
      abs_nclip: 3
      abs_sigma: 3.0
      abs_searchrad: 6.0
      abs_use2dhist: True
      abs_separation: 1.0
      abs_tolerance: 0.7
      sip_approx: True
      sip_max_pix_error: 0.01
      sip_degree: None
      sip_max_inv_pix_error: 0.01
      sip_inv_degree: None
      sip_npoints: 12
    skymatch:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      skymethod: match
      match_down: True
      subtract: False
      stepsize: None
      skystat: mode
      dqbits: ~DO_NOT_USE+NON_SCIENCE
      lower: None
      upper: None
      nclip: 5
      lsigma: 4.0
      usigma: 4.0
      binwidth: 0.1
    outlier_detection:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: False
      input_dir: ''
      weight_type: ivm
      pixfrac: 1.0
      kernel: square
      fillval: INDEF
      nlow: 0
      nhigh: 0
      maskpt: 0.7
      snr: 5.0 4.0
      scale: 1.2 0.7
      backg: 0.0
      kernel_size: 7 7
      threshold_percent: 99.8
      rolling_window_width: 25
      ifu_second_check: False
      save_intermediate_results: False
      resample_data: True
      good_bits: ~DO_NOT_USE
      allowed_memory: None
      in_memory: False
    resample:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: None
      search_output_file: True
      input_dir: ''
      pixfrac: 1.0
      kernel: square
      fillval: NAN
      weight_type: exptime
      output_shape: None
      crpix: None
      crval: None
      rotation: None
      pixel_scale_ratio: 1.0
      pixel_scale: None
      output_wcs: ''
      single: False
      blendheaders: True
      allowed_memory: None
      in_memory: True
    source_catalog:
      pre_hooks: []
      post_hooks: []
      output_file: None
      output_dir: None
      output_ext: .fits
      output_use_model: False
      output_use_index: True
      save_results: False
      skip: False
      suffix: cat
      search_output_file: True
      input_dir: ''
      bkg_boxsize: 1000
      kernel_fwhm: 2.141
      snr_threshold: 3.0
      npixels: 25
      deblend: False
      aperture_ee1: 30
      aperture_ee2: 50
      aperture_ee3: 70
      ci1_star_threshold: 2.0
      ci2_star_threshold: 1.8
2024-12-09 16:08:22,126 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:08:22,127 - stpipe.Image3Pipeline - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:08:22,127 - stpipe.Image3Pipeline - WARNING - 
2024-12-09 16:08:22,264 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw02739001002_02105_00001_nrca2_cal.fits' reftypes = ['abvegaoffset', 'apcorr']
2024-12-09 16:08:22,267 - stpipe.Image3Pipeline - INFO - Prefetch for ABVEGAOFFSET reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf'.
2024-12-09 16:08:22,267 - stpipe.Image3Pipeline - INFO - Prefetch for APCORR reference file is '/home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits'.
2024-12-09 16:08:22,269 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2024-12-09 16:08:22,281 - stpipe.Image3Pipeline - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jwst/associations/association.py:215: UserWarning: 'expname' contains path, but should only be a filename.  All input files should be in a single directory, so no path is needed.
2024-12-09 16:08:22,282 - stpipe.Image3Pipeline - WARNING -   warnings.warn(err_str, UserWarning)
2024-12-09 16:08:22,282 - stpipe.Image3Pipeline - WARNING - 
2024-12-09 16:08:23,037 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg running with args (<ModelContainer>,).
2024-12-09 16:08:25,052 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrca2_cal.fits.
2024-12-09 16:08:26,815 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00001_nrca4_cal.fits.
2024-12-09 16:08:28,740 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrca2_cal.fits.
2024-12-09 16:08:30,527 - stpipe.Image3Pipeline.tweakreg - INFO - Detected 200 sources in jw02739001002_02105_00002_nrca4_cal.fits.
2024-12-09 16:08:30,551 - stpipe.Image3Pipeline.tweakreg - INFO - 
2024-12-09 16:08:30,552 - stpipe.Image3Pipeline.tweakreg - INFO - Number of image groups to be aligned: 2.
2024-12-09 16:08:30,553 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:08:30,553 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() started on 2024-12-09 16:08:30.552994
2024-12-09 16:08:30,553 - stpipe.Image3Pipeline.tweakreg - INFO -       Version 0.8.9
2024-12-09 16:08:30,554 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:08:31,463 - stpipe.Image3Pipeline.tweakreg - INFO - Selected image 'GROUP ID: jw02739001002_02105_1' as reference image
2024-12-09 16:08:31,470 - stpipe.Image3Pipeline.tweakreg - INFO - Aligning image catalog 'GROUP ID: jw02739001002_02105_2' to the reference catalog.
2024-12-09 16:08:31,513 - stpipe.Image3Pipeline.tweakreg - INFO - Matching sources from 'jw02739001002_02105_00002_nrca' catalog with sources from the reference 'jw02739001002_02105_00001_nrca' catalog.
2024-12-09 16:08:31,513 - stpipe.Image3Pipeline.tweakreg - INFO - Computing initial guess for X and Y shifts...
2024-12-09 16:08:31,516 - stpipe.Image3Pipeline.tweakreg - INFO - Found initial X and Y shifts of 0.0007509, 0.000648 (arcsec) with significance of 288.9 and 299 matches.
2024-12-09 16:08:31,517 - stpipe.Image3Pipeline.tweakreg - INFO - Found 220 matches for 'GROUP ID: jw02739001002_02105_2'...
2024-12-09 16:08:31,518 - stpipe.Image3Pipeline.tweakreg - INFO - Performing 'shift' fit
2024-12-09 16:08:31,520 - stpipe.Image3Pipeline.tweakreg - INFO - Computed 'shift' fit for GROUP ID: jw02739001002_02105_2:
2024-12-09 16:08:31,520 - stpipe.Image3Pipeline.tweakreg - INFO - XSH: -2.40939e-05  YSH: -0.00118595
2024-12-09 16:08:31,521 - stpipe.Image3Pipeline.tweakreg - INFO - 
2024-12-09 16:08:31,521 - stpipe.Image3Pipeline.tweakreg - INFO - FIT RMSE: 0.00335582   FIT MAE: 0.00300579
2024-12-09 16:08:31,522 - stpipe.Image3Pipeline.tweakreg - INFO - Final solution based on 214 objects.
2024-12-09 16:08:31,590 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:08:31,590 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() ended on 2024-12-09 16:08:31.590084
2024-12-09 16:08:31,591 - stpipe.Image3Pipeline.tweakreg - INFO - ***** tweakwcs.imalign.align_wcs() TOTAL RUN TIME: 0:00:01.037090
2024-12-09 16:08:31,591 - stpipe.Image3Pipeline.tweakreg - INFO -  
2024-12-09 16:08:31,662 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS  274.714148067 -13.874666865 274.732194258 -13.875238842 274.732872320 -13.857853526 274.714793067 -13.857090476
2024-12-09 16:08:31,665 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:08:31,665 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use FITS instead.
2024-12-09 16:08:31,666 - stpipe.Image3Pipeline.tweakreg - WARNING -   warnings.warn(
2024-12-09 16:08:31,666 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:08:31,667 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:08:31,667 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use to_string() instead.
2024-12-09 16:08:31,667 - stpipe.Image3Pipeline.tweakreg - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:08:31,668 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:08:31,774 - stpipe.Image3Pipeline.tweakreg - INFO - Update S_REGION to POLYGON ICRS  274.714842292 -13.855929606 274.732930692 -13.856595359 274.733745223 -13.839137146 274.715569489 -13.838277336
2024-12-09 16:08:31,776 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/astropy/units/format/__init__.py:48: AstropyDeprecationWarning: The class "Fits" has been renamed to "FITS" in version 7.0. The old name is deprecated and may be removed in a future version.
2024-12-09 16:08:31,777 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use FITS instead.
2024-12-09 16:08:31,777 - stpipe.Image3Pipeline.tweakreg - WARNING -   warnings.warn(
2024-12-09 16:08:31,778 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:08:31,778 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2035: AstropyDeprecationWarning: The get_format_name function is deprecated and may be removed in a future version.
2024-12-09 16:08:31,778 - stpipe.Image3Pipeline.tweakreg - WARNING -         Use to_string() instead.
2024-12-09 16:08:31,779 - stpipe.Image3Pipeline.tweakreg - WARNING -   cunit = frame.unit[fidx].get_format_name(u.format.Fits).upper()
2024-12-09 16:08:31,779 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:08:31,818 - stpipe.Image3Pipeline.tweakreg - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/gwcs/wcs.py:2906: UserWarning: Double sampling check FAILED: Sampling may be too coarse for the distortion model being fitted.
2024-12-09 16:08:31,819 - stpipe.Image3Pipeline.tweakreg - WARNING -   warnings.warn(
2024-12-09 16:08:31,819 - stpipe.Image3Pipeline.tweakreg - WARNING - 
2024-12-09 16:08:31,842 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done
2024-12-09 16:08:32,078 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (<ModelContainer>,).
2024-12-09 16:08:32,201 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:08:32,201 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2024-12-09 16:08:32.201055
2024-12-09 16:08:32,202 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:08:32,202 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'match'
2024-12-09 16:08:32,203 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN
2024-12-09 16:08:32,203 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF
2024-12-09 16:08:32,204 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:08:32,204 - stpipe.Image3Pipeline.skymatch - INFO - ----  Computing differences in sky values in overlapping regions.
2024-12-09 16:08:34,055 - stpipe.Image3Pipeline.skymatch - INFO -    *  Group ID=1. Sky background of component images:
2024-12-09 16:08:34,056 - stpipe.Image3Pipeline.skymatch - INFO -       - Image ID=jw02739001002_02105_00001_nrca2_cal.fits. Sky background: 0.0258186
2024-12-09 16:08:34,056 - stpipe.Image3Pipeline.skymatch - INFO -       - Image ID=jw02739001002_02105_00001_nrca4_cal.fits. Sky background: 0.0258186
2024-12-09 16:08:34,057 - stpipe.Image3Pipeline.skymatch - INFO -    *  Group ID=2. Sky background of component images:
2024-12-09 16:08:34,058 - stpipe.Image3Pipeline.skymatch - INFO -       - Image ID=jw02739001002_02105_00002_nrca2_cal.fits. Sky background: 0
2024-12-09 16:08:34,058 - stpipe.Image3Pipeline.skymatch - INFO -       - Image ID=jw02739001002_02105_00002_nrca4_cal.fits. Sky background: 0
2024-12-09 16:08:34,059 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:08:34,059 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() ended on 2024-12-09 16:08:34.059335
2024-12-09 16:08:34,060 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() TOTAL RUN TIME: 0:00:01.858280
2024-12-09 16:08:34,060 - stpipe.Image3Pipeline.skymatch - INFO -  
2024-12-09 16:08:34,070 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch done
2024-12-09 16:08:34,305 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection running with args (<ModelContainer>,).
2024-12-09 16:08:34,312 - stpipe.Image3Pipeline.outlier_detection - INFO - Performing outlier detection on 4 inputs
2024-12-09 16:08:34,312 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter kernel: square
2024-12-09 16:08:34,313 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:08:34,313 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter fillval: INDEF
2024-12-09 16:08:34,313 - stpipe.Image3Pipeline.outlier_detection - INFO - Driz parameter weight_type: ivm
2024-12-09 16:08:34,314 - stpipe.Image3Pipeline.outlier_detection - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:08:34,366 - stpipe.Image3Pipeline.outlier_detection - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 16:08:34,440 - stpipe.Image3Pipeline.outlier_detection - INFO - 2 exposures to drizzle together
2024-12-09 16:08:36,292 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:08:38,960 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:08:40,020 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_outlier_i2d.fits
2024-12-09 16:08:40,030 - stpipe.Image3Pipeline.outlier_detection - INFO - 2 exposures to drizzle together
2024-12-09 16:08:42,466 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:08:45,743 - stpipe.Image3Pipeline.outlier_detection - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:08:46,794 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_outlier_i2d.fits
2024-12-09 16:08:46,918 - stpipe.Image3Pipeline.outlier_detection - INFO - Computing median
2024-12-09 16:08:57,871 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_outlier_i2d.fits
2024-12-09 16:08:57,898 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_outlier_i2d.fits
2024-12-09 16:08:57,902 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting median
2024-12-09 16:09:00,055 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 16:09:00,388 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_blot.fits
2024-12-09 16:09:02,515 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 16:09:02,854 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_blot.fits
2024-12-09 16:09:05,577 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 16:09:05,938 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_blot.fits
2024-12-09 16:09:08,670 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (2422, 4267)
2024-12-09 16:09:09,258 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_blot.fits
2024-12-09 16:09:09,259 - stpipe.Image3Pipeline.outlier_detection - INFO - Flagging outliers
2024-12-09 16:09:09,572 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 161 (0.00%)
2024-12-09 16:09:09,889 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 49 (0.00%)
2024-12-09 16:09:10,235 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 133 (0.00%)
2024-12-09 16:09:10,595 - stpipe.Image3Pipeline.outlier_detection - INFO - New pixels flagged as outliers: 65 (0.00%)
2024-12-09 16:09:10,621 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_blot.fits
2024-12-09 16:09:10,646 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_blot.fits
2024-12-09 16:09:10,672 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_blot.fits
2024-12-09 16:09:10,696 - stpipe.Image3Pipeline.outlier_detection - INFO - Removing file ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_blot.fits
2024-12-09 16:09:10,959 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca2_a3001_crf.fits
2024-12-09 16:09:11,214 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00001_nrca4_a3001_crf.fits
2024-12-09 16:09:11,494 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca2_a3001_crf.fits
2024-12-09 16:09:11,778 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/jw02739001002_02105_00002_nrca4_a3001_crf.fits
2024-12-09 16:09:11,779 - stpipe.Image3Pipeline.outlier_detection - INFO - Step outlier_detection done
2024-12-09 16:09:11,999 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (<ModelContainer>,).
2024-12-09 16:09:12,006 - stpipe.Image3Pipeline.resample - INFO - Driz parameter kernel: square
2024-12-09 16:09:12,007 - stpipe.Image3Pipeline.resample - INFO - Driz parameter pixfrac: 1.0
2024-12-09 16:09:12,007 - stpipe.Image3Pipeline.resample - INFO - Driz parameter fillval: NAN
2024-12-09 16:09:12,007 - stpipe.Image3Pipeline.resample - INFO - Driz parameter weight_type: exptime
2024-12-09 16:09:12,008 - stpipe.Image3Pipeline.resample - INFO - Output pixel scale ratio: 1.0
2024-12-09 16:09:12,060 - stpipe.Image3Pipeline.resample - INFO - Computed output pixel scale: 0.030776005311869827 arcsec.
2024-12-09 16:09:12,117 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for None
2024-12-09 16:09:12,663 - stpipe.Image3Pipeline.resample - INFO - Resampling science data
2024-12-09 16:09:14,592 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:17,340 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:20,650 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:23,904 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:24,746 - stpipe.Image3Pipeline.resample - INFO - Resampling variance components
2024-12-09 16:09:26,902 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:29,943 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:33,049 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:36,035 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:39,120 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:42,266 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:46,113 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:49,873 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:53,463 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:09:57,050 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:10:00,737 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:10:04,350 - stpipe.Image3Pipeline.resample - INFO - Drizzling (2048, 2048) --> (2422, 4267)
2024-12-09 16:10:05,375 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS  274.711011211 -13.874555187 274.732323507 -13.875342618 274.733750577 -13.838890827 274.712441621 -13.838103520
2024-12-09 16:10:06,091 - stpipe.Image3Pipeline.resample - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_sw_i2d.fits
2024-12-09 16:10:06,092 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2024-12-09 16:10:06,288 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (<ImageModel(2422, 4267) from image3_sw_i2d.fits>,).
2024-12-09 16:10:06,304 - stpipe.Image3Pipeline.source_catalog - INFO - Using APCORR reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_apcorr_0004.fits
2024-12-09 16:10:06,312 - stpipe.Image3Pipeline.source_catalog - INFO - Using ABVEGAOFFSET reference file: /home/runner/crds/references/jwst/nircam/jwst_nircam_abvegaoffset_0002.asdf
2024-12-09 16:10:06,313 - stpipe.Image3Pipeline.source_catalog - INFO - Instrument: NIRCAM
2024-12-09 16:10:06,313 - stpipe.Image3Pipeline.source_catalog - INFO - Detector: MULTIPLE
2024-12-09 16:10:06,314 - stpipe.Image3Pipeline.source_catalog - INFO - Filter: F200W
2024-12-09 16:10:06,314 - stpipe.Image3Pipeline.source_catalog - INFO - Pupil: CLEAR
2024-12-09 16:10:06,315 - stpipe.Image3Pipeline.source_catalog - INFO - Subarray: FULL
2024-12-09 16:10:06,336 - stpipe.Image3Pipeline.source_catalog - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/asdf/_asdf.py:189: AsdfWarning: copy_arrays is deprecated; use memmap instead. Note that memmap will default to False in asdf 4.0.
2024-12-09 16:10:06,337 - stpipe.Image3Pipeline.source_catalog - WARNING -   warnings.warn(
2024-12-09 16:10:06,337 - stpipe.Image3Pipeline.source_catalog - WARNING - 
2024-12-09 16:10:06,356 - stpipe.Image3Pipeline.source_catalog - INFO - AB to Vega magnitude offset 1.68628
2024-12-09 16:10:12,694 - stpipe.Image3Pipeline.source_catalog - INFO - Detected 4336 sources
2024-12-09 16:10:16,397 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote source catalog: ./nrc_im_demo_data/Obs001/stage3/image3_sw_cat.ecsv
2024-12-09 16:10:16,587 - stpipe.Image3Pipeline.source_catalog - INFO - Saved model in ./nrc_im_demo_data/Obs001/stage3/image3_sw_segm.fits
2024-12-09 16:10:16,588 - stpipe.Image3Pipeline.source_catalog - INFO - Wrote segmentation map: image3_sw_segm.fits
2024-12-09 16:10:16,593 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog done
2024-12-09 16:10:16,595 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline done
# Print out the time benchmark
time1 = time.perf_counter()
print(f"Runtime so far: {time1 - time0:0.0f} seconds")
print(f"Runtime for Image3: {time1 - time_image3:0.0f} seconds")
Runtime so far: 780 seconds
Runtime for Image3: 168 seconds

Verify which pipeline steps were run#

# Identify *_i2d file and open as datamodel
if doimage3:
    if do_swimage3:
        sw_i2d_file = os.path.join(image3_dir, f'{sw_product_name}_i2d.fits')
        i2d_sw_model = datamodels.open(sw_i2d_file)
        step_check_model = i2d_sw_model
        
    if do_lwimage3:
        lw_i2d_file = os.path.join(image3_dir, f'{lw_product_name}_i2d.fits')
        i2d_lw_model = datamodels.open(lw_i2d_file)
        step_check_model = i2d_lw_model

    # Check which steps were run. This should be the same regardless of whether
    # a sw or lw file is used.
    step_check_model.meta.cal_step.instance
{'assign_wcs': 'COMPLETE',
 'charge_migration': 'SKIPPED',
 'dark_sub': 'COMPLETE',
 'dq_init': 'COMPLETE',
 'flat_field': 'COMPLETE',
 'gain_scale': 'SKIPPED',
 'group_scale': 'SKIPPED',
 'ipc': 'SKIPPED',
 'jump': 'COMPLETE',
 'linearity': 'COMPLETE',
 'outlier_detection': 'COMPLETE',
 'persistence': 'SKIPPED',
 'photom': 'COMPLETE',
 'ramp_fit': 'COMPLETE',
 'refpix': 'COMPLETE',
 'resample': 'COMPLETE',
 'saturation': 'COMPLETE',
 'skymatch': 'COMPLETE',
 'superbias': 'COMPLETE',
 'tweakreg': 'COMPLETE'}

Check which reference files were used to calibrate the dataset

if doimage3:
    step_check_model.meta.ref_file.instance
{'area': {'name': 'crds://jwst_nircam_area_0261.fits'},
 'camera': {'name': 'N/A'},
 'collimator': {'name': 'N/A'},
 'crds': {'context_used': 'jwst_1293.pmap', 'sw_version': '12.0.8'},
 'dark': {'name': 'crds://jwst_nircam_dark_0424.fits'},
 'dflat': {'name': 'N/A'},
 'disperser': {'name': 'N/A'},
 'distortion': {'name': 'crds://jwst_nircam_distortion_0277.asdf'},
 'fflat': {'name': 'N/A'},
 'filteroffset': {'name': 'crds://jwst_nircam_filteroffset_0007.asdf'},
 'flat': {'name': 'crds://jwst_nircam_flat_0634.fits'},
 'fore': {'name': 'N/A'},
 'fpa': {'name': 'N/A'},
 'gain': {'name': 'crds://jwst_nircam_gain_0097.fits'},
 'ifufore': {'name': 'N/A'},
 'ifupost': {'name': 'N/A'},
 'ifuslicer': {'name': 'N/A'},
 'linearity': {'name': 'crds://jwst_nircam_linearity_0052.fits'},
 'mask': {'name': 'crds://jwst_nircam_mask_0076.fits'},
 'msa': {'name': 'N/A'},
 'ote': {'name': 'N/A'},
 'photom': {'name': 'crds://jwst_nircam_photom_0157.fits'},
 'readnoise': {'name': 'crds://jwst_nircam_readnoise_0266.fits'},
 'regions': {'name': 'N/A'},
 'saturation': {'name': 'crds://jwst_nircam_saturation_0097.fits'},
 'sflat': {'name': 'N/A'},
 'specwcs': {'name': 'N/A'},
 'superbias': {'name': 'crds://jwst_nircam_superbias_0220.fits'},
 'wavelengthrange': {'name': 'N/A'}}

8. Visualize the resampled images#

If you specified that the LW and SW outputs should be resampled onto the same pixel grid, you should be able to open the two i2d files and overlay them and see that the sources and pixel grids line up. If there is any misalignment, you may need to adjust tweakreg parameters in the calls to the Image3 pipeline in order to improve the alignment.

Below we use the Imviz tool within the jdaviz package to visualize the images, one filter at a time.

# Create an Imviz instance and set up default viewer for the F200W data
if doimage3 and do_swimage3:
    imviz_sw_i2d = Imviz()
    viewer_sw_i2d = imviz_sw_i2d.default_viewer

    # Read in the science array for our visualization dataset:
    i2d_sw_science = i2d_sw_model.data

    # Load the dataset into Imviz
    imviz_sw_i2d.load_data(i2d_sw_science)

    # Visualize the dataset:
    imviz_sw_i2d.show()

Remember that in this mosaic we have only two detectors: NRC2 and NRC4 (left and right, respectively). The dither is not large enough to cover the gap between the detectors, and so that gap is still visible in the mosaic.

if doimage3 and do_swimage3:
    viewer_sw_i2d.stretch = 'sqrt'
    viewer_sw_i2d.set_colormap('Viridis')
    viewer_sw_i2d.cuts = '95%'
# Create an Imviz instance and set up default viewer for the F444W data
if doimage3 and do_lwimage3:
    imviz_lw_i2d = Imviz()
    viewer_lw_i2d = imviz_lw_i2d.default_viewer

    # Read in the science array for our visualization dataset:
    i2d_lw_science = i2d_lw_model.data

    # Load the dataset into Imviz
    imviz_lw_i2d.load_data(i2d_lw_science)

    # Visualize the dataset:
    imviz_lw_i2d.show()
if doimage3 and do_lwimage3:
    viewer_lw_i2d.stretch = 'sqrt'
    viewer_lw_i2d.set_colormap('Viridis')
    viewer_lw_i2d.cuts = '95%'

Ovelaying the LW and SW images#

Let’s try putting the SW and LW images on top of one another to create a color image. This should work regardless of whether you resampled the two images onto the same pixel grid.

Let’s get the data first

if doimage3 and do_swimage3 and do_lwimage3:
    imviz_color = Imviz()
    viewer_color = imviz_color.default_viewer

    # Load the datasets into Imviz
    imviz_color.load_data(sw_i2d_file, data_label='sw')
    imviz_color.load_data(lw_i2d_file, data_label='lw')

    # Link images by WCS (without affine approximation)
    imviz_color.plugins['Links Control'].link_type = 'WCS'
    imviz_color.plugins['Links Control'].wcs_use_affine = False
2024-12-09 16:10:20,944 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:49: AstropyDeprecationWarning: The link_type function is deprecated and may be removed in a future version.
2024-12-09 16:10:20,945 - stpipe - WARNING -         Use align_by instead.
2024-12-09 16:10:20,945 - stpipe - WARNING -   exp_obj = getattr(self._obj, attr)
2024-12-09 16:10:20,946 - stpipe - WARNING - 
2024-12-09 16:10:21,651 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:49: AstropyDeprecationWarning: The wcs_use_affine function is deprecated and may be removed in a future version.
2024-12-09 16:10:21,652 - stpipe - WARNING -         Use wcs_fast_approximation instead.
2024-12-09 16:10:21,652 - stpipe - WARNING -   exp_obj = getattr(self._obj, attr)
2024-12-09 16:10:21,653 - stpipe - WARNING - 
2024-12-09 16:10:21,653 - stpipe - WARNING - /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/jdaviz/core/user_api.py:85: AstropyDeprecationWarning: The wcs_use_affine function is deprecated and may be removed in a future version.
2024-12-09 16:10:21,653 - stpipe - WARNING -         Use wcs_fast_approximation instead.
2024-12-09 16:10:21,654 - stpipe - WARNING -   return setattr(self._obj, attr, value)
2024-12-09 16:10:21,655 - stpipe - WARNING - 

Now define some options to make the picture look nice.

# Set the colors for the two images. 
if doimage3 and do_swimage3 and do_lwimage3:
    plot_options = imviz_color.plugins['Plot Options']
    plot_options.image_color_mode = 'Color'
    img_settings = {'sw': {'image_color': '#61d3e1',
                           #'stretch_vmin': 0,
                           #'stretch_vmax': 4,
                           #'image_opacity': 0.32,
                           #'image_contrast': 0.69,
                           #'image_bias': 0.39
                           },
                    'lw': {'image_color': '#ff767c',
                           #'stretch_vmin': 0,
                           #'stretch_vmax': 16,
                           #'image_opacity': 0.4,
                           #'image_contrast': 0.94,
                           #'image_bias': 0.74
                           }
                    }

Populate the imviz instance with the settings in the cell above and visualize the dataset

# Now populate the imviz instance with the settings in the cell above.
if doimage3 and do_swimage3 and do_lwimage3:
    for layer, settings in img_settings.items():
        plot_options.layer = f'{layer}[DATA]'
        for k, v in settings.items():
            setattr(plot_options, k, v)
# Visualize the dataset
if doimage3 and do_swimage3 and do_lwimage3:
    imviz_color.show()

9. Visualize Detected Sources#

Using the source catalogs created by the Image3 stage of the pipeline, mark the detected sources, using different markers for point sources and extended sources. The source catalogs are saved in image3/image3_sw_cat.ecsv and image3/image3_lw_cat.ecsv. This time, we will provide the i2d filename to the imviz load_data function, rather than just the array of pixel values. This way, imviz will be able to make use of the WCS in the file. This will allow the sources in the source catalog to be accurately marked in the display.

Read in catalog file and identify point/extended sources#

if doimage3:
    if do_swimage3:
        sw_catalog_file = sw_i2d_file.replace('i2d.fits', 'cat.ecsv')
        sw_catalog = Table.read(sw_catalog_file)
    
        # To identify point/extended sources, use the 'is_extended' column in the source catalog
        sw_pt_src, = np.where(~sw_catalog['is_extended'])
        sw_ext_src, = np.where(sw_catalog['is_extended'])
    
        # Define coordinates of point and extended sources
        sw_pt_coord = Table({'coord': [SkyCoord(ra=sw_catalog['sky_centroid'][sw_pt_src].ra,
                                                dec=sw_catalog['sky_centroid'][sw_pt_src].dec)]})
        sw_ext_coord = Table({'coord': [SkyCoord(ra=sw_catalog['sky_centroid'][sw_ext_src].ra,
                                                 dec=sw_catalog['sky_centroid'][sw_ext_src].dec)]})

    if do_lwimage3:
        lw_catalog_file = lw_i2d_file.replace('i2d.fits', 'cat.ecsv')
        lw_catalog = Table.read(lw_catalog_file)

        # To identify point/extended sources, use the 'is_extended' column in the source catalog
        lw_pt_src, = np.where(~lw_catalog['is_extended'])
        lw_ext_src, = np.where(lw_catalog['is_extended'])

        # Define coordinates of point and extended sources
        lw_pt_coord = Table({'coord': [SkyCoord(ra=lw_catalog['sky_centroid'][lw_pt_src].ra,
                                                dec=lw_catalog['sky_centroid'][lw_pt_src].dec)]})
        lw_ext_coord = Table({'coord': [SkyCoord(ra=lw_catalog['sky_centroid'][lw_ext_src].ra,
                                                 dec=lw_catalog['sky_centroid'][lw_ext_src].dec)]})

Mark the extended and point sources on the images#

Display the image with sources indicated by circles. Point sources will be marked by small pink circles and extended sources will be marked by white circles. Looking at the entire mosaic, there are so many sources found that it’s hard to see much of anything. To get a clearer view, try zooming in on various areas using the magnifying glass icon on the banner immediately above the image.

First we visualize the data without the point sources.

# Read in SW i2d file to Imviz
if doimage3 and do_swimage3:
    imviz_sw_cat = Imviz()
    viewer_sw_cat = imviz_sw_cat.default_viewer
    imviz_sw_cat.load_data(sw_i2d_file)

    # Adjust settings for viewer
    viewer_sw_cat.stretch = 'sqrt'
    viewer_sw_cat.set_colormap('Viridis')
    viewer_sw_cat.cuts = '95%'

    imviz_sw_cat.show()

Now we add the point sources

# Add marker for point sources:
if doimage3 and do_swimage3:
    viewer_sw_cat.marker = {'color': 'pink', 'markersize': 50, 'fill': False}

    viewer_sw_cat.add_markers(sw_pt_coord, use_skycoord=True, marker_name='point_sources')

    # Add marker for extended sources:
    viewer_sw_cat.marker = {'color': 'white', 'markersize': 100, 'fill': False}

    viewer_sw_cat.add_markers(sw_ext_coord, use_skycoord=True, marker_name='extended_sources')

We do the same with the LW file. First we visualize the data.

# Repeat using the LW file
if doimage3 and do_lwimage3:
    imviz_lw_cat = Imviz()
    viewer_lw_cat = imviz_lw_cat.default_viewer
    imviz_lw_cat.load_data(lw_i2d_file)

    # Adjust settings for viewer
    viewer_lw_cat.stretch = 'sqrt'
    viewer_lw_cat.set_colormap('Viridis')
    viewer_lw_cat.cuts = '95%'

    imviz_lw_cat.show()

Now we mark the point sources

# Add marker for point sources:
if doimage3 and do_lwimage3:
    viewer_lw_cat.marker = {'color': 'pink', 'markersize': 50, 'fill': False}

    viewer_lw_cat.add_markers(lw_pt_coord, use_skycoord=True, marker_name='point_sources')

    # Add marker for extended sources:
    viewer_lw_cat.marker = {'color': 'white', 'markersize': 100, 'fill': False}

    viewer_lw_cat.add_markers(lw_ext_coord, use_skycoord=True, marker_name='extended_sources')

10. Notes#

  • Note that the strategy presented in this notebook for placing the SW data onto the same pixel grid as the LW data can be applied to data from any two datasets, regardless of filter or channel. By saving the gWCS from the first dataset into an asdf file and providing that file to the Image3 call with the second dataset, the resulting i2d images will be aligned onto the same pixel grid.

  • If you notice poor alignment across tiles within a single i2d image, or between i2d images that you expect to be aligned, try adjusting the parameters in the tweakreg step. With these, you can customize which sources tweakreg identifies and uses for the alignment.


stsci_logo